How to Choose the Right Tools for Your Internal Developer Platform

Platform engineering is a relatively new discipline around building internal tools that accelerate and standardize the software delivery process with the goal of boosting developer productivity. It describes the creation and ongoing support of an internal developer platform, or IDP, as an abstraction that centralizes the tools developers need to build, test and ship their work. The IDP and the process of its creation is treated in much the same way as a commercial software product is treated, with the organization’s software engineers being its target customers.

IDPs increase delivery throughput by granting developers more autonomy. They can use the platform's tools to perform self-service actions, such as starting a new environment or safely accessing production environment logs. IDPs also promote standardization of developer toolchains, ensuring everyone follows one consistent process to deliver changes. This makes it less likely that bugs, regressions and compliance breaches will occur.

IDPs aren't typically created entirely from scratch; there's an expansive ecosystem of tools you can integrate to assemble one without having to build too much yourself. Nonetheless, you should fully evaluate each option to ensure it's a good fit for your engineering requirements.

This guide will help you make strategic IDP tool selections that set you up to win at platform engineering.

Strategic Tool Selection for Your IDP

An IDP should be unique to your organization and how it works; no two IDPs will include the same set of tools. Finding the right utilities for your platform is an exercise in thoughtfully crafting evaluation criteria that allow you to easily shortlist compelling options and discard unsuitable alternatives.

This process begins with identifying the capabilities your IDP needs to offer, in addition to the ways in which users will interact with available features (such as via a CLI or web application). The platform team should communicate with developers to learn where problems are occurring in the software delivery process. These pain points are the priorities your IDP should solve, so they'll inform your initial tool selection.

Other stakeholders should be involved too: DevOps and SRE teams, AppSec experts, incident management specialists and product managers can all benefit from the improvements in process standardization and visibility that IDPs provide. Discussing the day-to-day challenges encountered by these groups will reveal further capabilities to add to your platform.

By the end of this first stage, you should know who your user personas are and what they seek from your IDP. For example, your personas could be a developer who wants to be able to deploy the code from their IDE to a new staging environment on demand and a product manager who needs to know when a deployment last went live and what it changed. You can use these high-level requirements to start identifying specific tools and processes that'll power your IDP.

Identify the Tools Your IDP Requires

Your IDP's components should span the complete DevOps life cycle, from building code to testing and deploying it using automated processes such as CI/CD. A comprehensive IDP will include many different tools that collectively orchestrate your workflows and grant autonomy to developers.

Let's look at some of the key tool categories that are commonly integrated into an IDP.

Essential Features for Every IDP

These features are the basics that your entire userbase is likely to interact with. They can be found in most IDPs since they're fundamental participants in the modern software delivery life cycle.

• Version control: A version control system (VCS) such as GitHub, GitLab or Bitbucket allows devs to manage codebase changes and seamlessly collaborate with their peers. IDPs can expose controls that simplify VCS interactions, such as creating a new pull request or forking a repository into a new environment.
• Self-service environment provisioning: Infrastructure as code (IaC) tools such as Terraform, Ansible and Pulumi can be integrated into IDPs to help developers provision new infrastructure on demand and make the testing process more efficient.
• IDEs: Devs need a code authorship IDE that acts as a hub for the development process. Building IDP plugins for popular IDEs such as Visual Studio, Xcode and JetBrains lets devs interact with the platform's features while avoiding a context switch into another tool.
• Build tools: IDPs can simplify complex build processes by offering convenient tools capable of producing builds for every platform you target. For example, a developer could use the IDP to start a Jenkins pipeline that runs a cross-platform build and then delivers the output binaries to the dev's workstation.
• Collaboration platforms and documentation portals: IDPs can help coordinate interteam communication and facilitate knowledge sharing. Try exposing the IDP as chat commands in apps like Slack and Microsoft Teams to let team members quickly request helpful context when they need it.
• Software service catalog: IDPs should provide developers with access to a catalog of software services and automated processes they can run within their workflows. Platforms such as Backstage, Qovery and Port can be used to create self-service hubs and portals that present your service catalog as one unified interface.
• Built-in security capabilities: IDPs help enable continuous security policy enforcement. Ensuring all devs use the IDP to achieve their tasks means security checks can be baked in, such as by running SAST and SCA checks using tools like Anchore and Trivy whenever code is changed or an environment is started.

Once you've selected tools that cover these foundations, you can start layering over the additional features that are specific to your team's requirements.

Specialized Tools and Features for Advanced Use Cases

The following categories are suggestions of advanced features that developers benefit from accessing via an IDP. You won't necessarily need to include all these functions because their applicability is dependent on the types of development activities you complete.

• Container orchestration management: IDPs can provide abstractions for container orchestration technologies such as Kubernetes, making deployment operations more accessible to developers building microservices.
• Multicloud management and optimization: Multicloud and hybrid cloud workflows can be difficult to manage, but the centralization enabled by IDPs makes it easier to ensure consistent configuration across environments and monitor what's running. You can use tools such as Rancher, OpenStack and Control Plane to simplify developer access to these resources.
• API management tools: API-first development requires developers to be able to easily test deployed endpoints and check that they're performing as expected. Solutions such as Kong, Tyk and Postman help manage API development and are ideal to integrate into developer portals.
• Configuration management as code: Configuration as code (CaC) and centralized configuration management techniques are close relatives of IaC and GitOps. Systems such as Puppet let you automate configuration rollouts and then detect when discrepancies occur.
• DevOps loop automation: Improved DevOps automation is one of the main benefits of IDPs, but the specific capabilities you need will depend on how your team works. Automation is primarily achieved by providing one place to manage CI/CD pipelines executed in services such as GitHub Actions and GitLab CI/CD, alongside business-specific automated workflows implemented in tools like Apache Airflow and Argo Workflows.
• VM hypervisor management: Teams that run their workloads in VMs can benefit from IDPs that consolidate common hypervisor functions, such as by interfacing with tools like Hyper-V and VMware to expose details of running VMs and provide options to create new ones.
• Database management and automation tools: IDPs can make database management operations more accessible by providing one-click workflows to initiate backups, provision more replicas or seed test data for development. Integrate with managed cloud database services such as Google Cloud databases and Azure SQL Database to let devs interact with databases while preserving security guardrails.
• Real-time data processing: IDPs can also help devs make sense of real-time data streams. You could create catalog services that report trends in the data captured by tools like Apache Kafka and Apache Hadoop, for example.
• Observability, monitoring and alerting: Observability suites such as Prometheus, Grafana and the Elastic Stack (ELK) are prime candidates to include in an IDP. Providing key metrics and dashboards within a developer portal interface keeps all stakeholders continually informed of DevOps performance and any issues that occur.
• Service mesh solution: Service meshes such as Istio and Linkerd simplify networking for microservices deployments by managing service-to-service communication. Your IDP could expose service mesh capabilities so developers can quickly publish new services directly from their terminal or IDE.
• Feature flagging capabilities: High-performing DevOps teams ship changes fast, sometimes before they're ready for widespread use. Feature flag tools such as Flagsmith, Flipt and Flipper Cloud allow devs to easily hide new capabilities in live deployments; providing access to these services as simple IDP commands means developers have fewer third-party platforms to learn.
• Continuous compliance management: The consistency advantages of IDPs support continual compliance management. You can be sure that developers are only using approved tools to achieve their tasks, thus reducing the risk of compliance breaches.

This list isn't meant to be exhaustive; there are plenty more features that can be useful within an IDP. If your developers regularly have to complete specific tasks as part of their role, then it makes sense to build those processes into the platform. This could range from simple facilities that alert developers when a new customer ticket is opened to more complex automation (such as for AI/ML model training).

How to Select the Right Tools

After you've identified which tools are candidates to include in your IDP, it's time to create a decision tree that guides you toward choosing the best option in each category. This is where you assess each tool for compatibility with your internal priorities, such as budget constraints and efficient utilization of available resources.

Documenting decisions you make—both why you chose one tool and didn't choose an alternative—is advisable so future platform engineers can understand how your IDP has evolved. Similarly, it's important to record what your priorities actually are based on discussions held with developers and stakeholders. Reviewing this documentation periodically can help you gauge whether implementation efforts are proceeding in line with your plan.

Infrastructure and Resource Constraints

Some tools won't be suitable for your IDP due to specific resource requirements that you're unable to meet. For example, a tool with substantial CPU, memory, storage or networking demands might be impossible to accommodate if you're looking for something you can self-host on existing hardware.

This factor can appear when evaluating weighty source control and CI/CD servers. It's also a frequent issue when considering data processing and AI/ML workloads because they're usually highly compute-intensive.

Budget Constraints

Budgetary issues can make even the most appealing tools impractical. You need to check which license tier you require, how many seats must be purchased and whether there are extra costs for additional features or support agreements. Options and policies vary substantially by provider—Pulumi's IaC service requires a paid plan if you need more than one team member, for example, while Puppet is open source but also offers an enterprise plan with extra support, scalability and compliance management features.

Even where tools are free and open source, it's worth assessing how much time you'll need to spend integrating them with your platform. Each hour spent preparing a tool can be a significant cost. It could sometimes work out more cost-effectively to use a fully managed service that comes with a hands-off support option, such as paying for cloud CI/CD minutes on GitHub or GitLab instead of maintaining your own Jenkins server.

Security and Compliance Requirements

Compatibility with relevant security and compliance standards is a must for any tool you add to your DevOps workflow. But no two organizations have quite the same requirements, so it's vital you clearly identify yours. For example, tools that are going to be exposed to critical PII, financial or healthcare data need to demonstrate compliance with applicable regulations such as GDPR and HIPAA, even if you perceive your organization to be operating at a relatively small scale.

Need ISO 27001 compliance? That might exclude some smaller open source tools and push you towards established SaaS providers. Similarly, your choices could be restricted if your IT team requires vendors to publish a software bill of materials (SBOM) for their services.

Scalability and Performance Requirements

IDP tools need to be scalable so they can grow as your apps and development teams expand. Contrary to the purpose of an IDP, using tools that lack adequate scalability may actually cause future productivity issues by preventing developers from innovating efficiently. Performance issues in one service can also have cascading impacts on the other components in your IDP.

Choosing IDP tools that are modular, cloud native and designed around the needs of enterprise customers makes it more likely you'll be able to scale successfully. You should assess the general performance of each tool—such as its responsiveness and whether it accelerates or delays your workflows—then evaluate how easily you could integrate it with a different set of services in the future, in case your stack changes later.

SaaS IDP solutions such as Qovery and Port make it easy to get started, but building your own system with custom technologies or a framework like Backstage could prove easier to scale as your service fleet increases. Similarly, while self-hosting your tools may initially be attractive—you can access high-performance hardware without the ongoing costs of cloud resources—you might find yourself unable to scale up to the requirements of more demanding workloads in the future.

Standardization and Consistency

DevOps life cycle standardization is one of the main outcomes of IDP adoption. Achieving good consistency depends on your tools being compatible with one another and developers having easy access to your service catalog. This ensures devs will be able to use the platform to attain their objectives without having to reach for other tools.

You'll have to go back and reevaluate your shortlist if you find the selected tools can't be connected or the complexity would render it unviable. Issue trackers, source control providers, CI/CD servers and IaC solutions can usually integrate with one another, for example, but different options can be easier to with specific neighbors. Bitbucket's source control solution is designed to work with Jira and Bitbucket Pipelines, while GitLab bundles its own CI/CD service but also provides limited support for external alternatives.

Similarly, you'll need to check that your tech stack is compatible with any managed PaaS services you deploy to. Platforms like Heroku help simplify cloud operations but also restrict the programming languages and frameworks you can choose from.

Standardization also depends on obtaining organizational buy-in for your IDP. No matter how great your tools are, some devs may still want to work in ways that are already familiar to them, while others could be unaware of your platform's capabilities. Communication, documentation and use of the platform team as evangelists will help bring all devs into the fold, producing a more predictable DevOps process where changes are developed using one consistent method.

Analyze Outcomes to Iterate on Your Toolchain

Once you've completed the steps discussed above, you'll have created a list of viable tools to potentially include in your IDP. You'll also have identified tools you've decided to exclude because they don't meet your requirements in areas such as budget, infrastructure constraints and compliance.

You can now use the viable tools to build your initial IDP implementation. The concrete steps you'll have to follow will be specific to the tools you're using and the workflows that your developers require. Nonetheless, it's usually a case of connecting APIs, CI/CD pipelines and developer portal solutions together to form the service catalog that developers will interact with.

You may find your IDP solves all your DevOps problems, but this is unlikely on your first attempt. In reality, you'll probably find some tools aren't as easy to use as expected or fail to scale with changes in your development requirements. You should regularly analyze your IDP's performance based on feedback from developers and collected metrics, then iterate on your tool selection to apply targeted optimizations.

Appraising an IDP's success must involve communication with the developers who use it; platform teams can easily be misled into thinking an IDP is effective because it's solving standardization, security and compliance concerns, but developers could still be encountering productivity roadblocks due to missing functionality or feedback loops taking too long to complete. It's therefore vital to continually converse with all stakeholders to ensure they're receiving what they need across the key priorities of productivity, scalability, security, consistency and cost.

Conclusion

This article explained which tool categories should feature in your IDP and how you can evaluate specific options to find the best tool for you. A successful IDP implementation will support your efforts to increase software delivery throughput and enhance developer satisfaction, but this depends on tools being chosen strategically. Don’t be swayed by what's popular today, as those options won't necessarily align with your platform requirements.