
\ I pulled the Google Analytics data on a Thursday afternoon, mostly because I was preparing a slide deck for the engineering all-hands to celebrate our Q2 platform adoption. The numbers were brutal. In month one, we had 140 unique engineers log into our new Internal Developer Platform (IDP). They clicked around the catalog, generated a few scaffolded services, and generally did the things you do when leadership tells you a new tool is the future of the company. In month two, that number dropped to 52. By month three—the week I was pulling the data—we had exactly 28 unique users. Most of them were on my team (the platform team), checking if the IDP was still running. A full 80% of the engineering org had tried our "Spotify-like" developer experience, quietly decided it was worse than what they had before, and gone back to copy-pasting Dockerfiles from legacy repositories. We had spent six months of dedicated engineering time building this. Four senior engineers. We had custom plugins, a beautifully themed UI, and a mandate from the VP of Engineering. And it was completely, undeniably dead. The Mirage of the "Out-of-the-Box" Platform To understand why it failed, you have to understand what we thought we were building. We had roughly 40 microservices, a growing EKS cluster, and a severe onboarding problem. It took a new hire three weeks to get a service from their laptop to a staging environment. They had to know Terraform, Helm, GitHub Actions, and exactly which senior engineer to bribe in Slack to get an IAM role provisioned. The industry promised us that an Internal Developer Platform was the answer. We chose Backstage. It’s open-source, extensible, and has a massive ecosystem. But here is the trap we fell into, and the one I see dozens of other platform teams falling into today: Backstage is a developer portal, not a developer platform. A portal is a pane of glass. It is a UI that aggregates information. A platform is the underlying engine that actually provisions infrastructure, manages state, and enforces policy. We stood up the pane of glass, but we didn't build the engine behind it. When a developer clicked "Create New Service" in our shiny new UI, they thought they were getting a running container in Kubernetes. What they actually got was a GitHub repo with a Hello World Node app, a broken CI pipeline because the secret wasn't provisioned, and a Jira ticket automatically assigned to the DevOps queue asking for AWS permissions. We hadn't reduced cognitive load. We just put a React frontend over the exact same friction. The YAML Tax That Broke the Camel's Back If the lack of actual automation was the underlying disease, the metadata model was the symptom that drove engineers away. To get your service to show up in the catalog, you had to write a catalog-info.yaml file. This is standard Backstage practice. But because we wanted to track everything —ownership, PagerDuty integration, API dependencies, lifecycle state—our required YAML file grew. This is the actual template we forced engineers to commit to the root of every repository: # catalog-info.yaml apiVersion: backstage.io/v1alpha1 kind: Component metadata: name: payment-processor description: Handles Stripe webhooks and ledger updates tags: - nodejs - payments - tier-1 links: - url: https://datadoghq.com/dashboard/payments title: Datadog Dashboard icon: dashboard - url: https://wiki.mycompany.com/payments title: Runbook icon: docs annotations: github.com/project-slug: myorg/payment-processor backstage.io/techdocs-ref: dir:. pagerduty.com/integration-key: ${PAGERDUTY_KEY} sonarqube.org/project-key: payment-processor spec: type: service lifecycle: production owner: group:payment-squad system: core-billing providesApis: - payment-api consumesApis: - user-profile-api - ledger-api dependsOn: - resource:stripe-db It doesn't look terrible until you realize that developers had to maintain this manually. When an API changed names? Update the YAML. When the team reorganized? Update the YAML across 12 repos. If you made a typo in the owner field, the YAML parser would silently fail, your service would vanish from the catalog, and you wouldn't know until the platform team pinged you two weeks later. We asked engineers why they stopped using the portal. One senior backend developer told me, "I spend my day writing business logic. I am not going to spend 20 minutes debugging YAML whitespace just so my service has a nice icon on a dashboard I never look at." He was entirely right. We had shifted the burden of platform maintenance onto the users. The Pivot: From UI to API When the 80% abandonment metric forced our hand, we had to make a choice. We could mandate usage (which never works), or we could scrap the portal and build what the engineers actually needed. We scrapped the portal. We realized that our engineers didn't want a UI. They lived in their terminals and their IDEs. They didn't want to click a button in a browser to create a service; they wanted to run a CLI command and have it actually work . We spent the next two months building a lightweight Go CLI and a rudimentary backend orchestrator. We called it pave . Here is what creating a service looked like after the pivot: $ pave create service \ --name payment-processor \ --template node-express \ --team payment-squad \ --env production [+] Creating GitHub repository myorg/payment-processor... [+] Bootstrapping Node.js Express template... [+] Provisioning AWS IAM Roles for EKS... [+] Creating Postgres database stripe-db via Terraform Cloud... [+] Injecting secrets to GitHub Actions... [+] Registering service in internal catalog... Service 'payment-processor' is live. Repo: https://github.com/myorg/payment-processor CI/CD: https://github.com/myorg/payment-processor/actions Notice what is missing? The YAML. The pave backend handled the metadata generation. It automatically wired up the Datadog dashboards via API. It automatically provisioned the PagerDuty service. It generated the catalog metadata dynamically based on the Git repository state and the AWS tags. We stopped asking developers to describe their services to the platform, and started making the platform smart enough to discover the services itself. The Three Rules of Platform Engineering If you are a platform engineer looking at the hype cycle of 2025, it is incredibly tempting to reach for a massive, feature-rich portal framework to solve your DevEx problems. But an interface over a broken process is just a shinier broken process. Through our failure, we established three rules for our platform team that we still use today: 1. Paved Roads, Not Toll Booths A developer platform should provide a "paved road"—a path of least resistance that handles the heavy lifting (IAM, CI/CD, secret management, routing) automatically. If your platform requires developers to learn a new metadata schema, write extra YAML, or manage catalog state, it is a toll booth. Engineers will drive around it. 2. Automation First, UI Second Do not build a web interface until the underlying automation is flawless. If a developer cannot provision a service from end-to-end using a simple shell script or an API call, a React frontend is not going to save you. Build the orchestrator. Then build the CLI. Only build the UI when non-technical stakeholders (product managers, QA) need visibility. 3. State Must Be Discovered, Not Declared Never force a human to manually type out dependencies that a machine can infer. If Service A talks to Service B, your platform should discover that via network tracing (e.g., eBPF or Istio) or code analysis, not by asking a developer to remember to add consumesApis: [service-b] to a YAML file. Humans drift; discovered state does not. The Aftermath We still use a catalog today, but it is entirely read-only and automatically generated. Our engineers never touch it. They use the CLI. Our "time to first deployment" dropped from three weeks to 12 minutes. That isn't because we built a beautiful dashboard. It's because we stopped trying to be Spotify, and started actually automating the miserable parts of our engineers' jobs. If you are planning to spend the next six months standing up a developer portal, stop. Ask your engineers if they want a portal, or if they just want their database credentials to work on the first try. I promise you, it's the latter. The pave CLI we built eventually evolved into a thin wrapper around our internal Terraform modules and GitHub Actions API. If you're building something similar, start with the APIs you already have. \
View original source — Hacker Noon ↗


