Overview of DevOps and Infrastructure as Code eBook
In the not so distant past, code was tightly coupled with the infrastructure it ran on. This meant software development cycles were long, changes were difficult to make, and scaling to meet demand was difficult and expensive.
Today, developers need to make faster code changes, have fewer hardware dependencies, and scale applications rapidly to meet changing business demands. With the trend towards microservices as the solution to the problems of monolithic application development, developers want to be able to provision, configure, test, and manage application development and infrastructure programmatically.
DevOps enables teams to release early and often. It involves:
A transformation in culture to move away from monolithic to microservices frameworks
A growing trend towards using infrastructure as code (IaC) to automate the deployment of infrastructure resources
The right tools to provision, configure, build, test, and manage infrastructure
DevOps has brought about a change in not only tools, but the culture of software development – Agile development. With DevOps, building, testing, and releasing software can happen more frequently with the aid of automation.
But, in addition to automating application and platform, agile development can be further facilitated through the use of automation in infrastructure management, i.e. infrastructure agility. Consequently, DevOps has evolved to include Infrastructure as Code (IaC) frameworks to fulfill this need. These frameworks provide declarative management of resources, thereby removing tasks such as typing commands in in command line interfaces (CLIs) and configuring infrastructure manually.
For those that have adopted or are considering making the change to DevOps, there can be a great deal of confusion around the DevOps toolchain. The following section explores this topic further.
Understanding the DevOps Toolchain
Moving to a DevOps culture inevitably means adopting a DevOps toolchain. A DevOps toolchain is a group of tools organized around the stages of the software development process.
Development teams will adopt specific tools within the toolchain that fit their software development process. At each stage of the software development process, you’ll find several tools that can be used. You’ll also find that many of these tools are flexible and can be used at different stages and in different capacities.
The DevOps toolchain begins with development environments and source code control. Developers write code in development environments that provide the language support and other features that they need, often integrating with version control systems (VCS) or source control management (SCM). Source code management and versioning tools keep track of the changes to application code, providing a single source of truth about an application and its history.
In a DevOps culture, source control tools often do more than just handle revision control, review, and merging of application code.
DevOps practitioners frequently use source code control tools to maintain control of documentation, configurations, information, databases, and IaC projects, such as Ansible playbooks, Chef cookbooks, and so on.
Popular tools: Git, Subversion, Perforce, Bitbucket, Developer Cloud Service
Developers are constantly making changes to the code – and so are other developers on the team. The codebase changes almost continually, and there is no time to wait for code integration and testing to take place days or weeks later.
Continuous integration (CI) tools automate the building and testing of applications as the code changes. Automated test execution provides a fast feedback loop and developers are encouraged to make commits more frequently to source control management. In addition, builds happen automatically and problems with the shared codebase can be identified quickly.
Popular build frameworks: Maven, Gradle, Ant, npm, Grunt, Bower, Gulp
Popular CI tools: Jenkins, Hudson, GitLab CI, Travis CI, Bamboo, Oracle Developer Cloud Service, Oracle Container Pipelines
Provisioning tools prepare the infrastructure needed for an application to go into service. Provisioning tools may create VMs, containers, or spin up bare metal servers, depending upon the needs of the application. Provisioning tools will handle compute, networking, and storage configuration, as well as make a server ready for service.
Automated provisioning with Infrastructure as Code enables self-service deployment of computing resources without having to wait for IT to manually provision servers and other infrastructure.
Further, IaC using open standards like Terraform lets you standardize your infrastructure as code engine and create repeatable environments in hybrid and multi cloud environments.
Popular tools: Docker , Terraform, Chef, Puppet, Ansible, Salt, Vagrant, Resource Manager (Oracle Cloud service for IaC)
With continuous integration (CI), testing no longer happens at the end of a long release cycle, but must happen as new builds appear. Continuous testing provides feedback on the business risks of the build being tested, to determine whether the build is too risky to proceed. This process validates both functional and non-functional requirements of the application, as well as integrations spanning system boundaries.
Testing tools must be able to accurately assess and identify problems at each layer and be flexible enough to adapt to the fast-paced changes in modern applications. In addition, user experience (UX) tests should be part of the continuous testing process.
Popular tools: Cucumber, JUnit, Selenium
Once a build moves past the testing stage of the software development pipeline, it needs to be packaged and staged prior to release. Packages include all of the artifacts of the application, the changes that need to be performed, as well as prerequisite checks, and post-installation actions.
Software packaging tools, or universal package managers, help decouple the application from the infrastructure, so that deployment can be automated across the environment and all dependencies will be included in the package.
Popular tools: JFrog Artifactory, Sonatype Nexus Repository, Inedo Proget, Developer Cloud Service, Oracle Cloud Infrastructure Registry.
After software has been packaged and staged for release, another set of tools in the DevOps toolchain take over the release management. Release management involves several different activities – scheduling and orchestrating the release, deployment of the application, and handling deployment errors.
Continuous delivery (CD), which complements continuous integration (CI), enables the release of software frequently and reliably.
Application Release Automation (ARA) tools perform release management by creating consistent, repeatable deployments of software across a variety of environments.
These tools assist with CD by eliminating the need for custom deployment processes and scripts for applications, reducing configuration errors in the process. This means releases become more predictable and reliable with ARA tools.
Popular tools: XL Release, Automic, Clarive, uDeploy, Developer Cloud Service
Configuration management tools automate the process of ensuring that deployed applications and infrastructure are in the desired state. This may involve performing additional infrastructure or application configurations, configuring users, groups, and other applications on the software, and configuring networking and storage.
Master images of the desired state of infrastructure may be created for automated deployment. Both continuous configuration automation (CCA) and IaC tools fall into this segment of the DevOps toolchain.
Popular tools: Chef, Puppet, Ansible, Terraform, Salt
Application Performance Management (APM) tools provide continuous monitoring of applications once they are released into production and configuration is complete. APM solutions monitor the performance of the application and the user experience.
Continuous monitoring is crucial to ensuring that users have a positive experience with the application, that service level agreement (SLA) levels are met, and that application issues in the production environment are identified, diagnosed, and corrected quickly.
Log management tools provide log collection, aggregation, analysis, search, and reporting.
These tools also provide insight into the runtime behaviors of the system, thereby facilitating continuous improvement and DevOps. Security and compliance are one of the primary drivers for log management.
Popular tools: ELK, Splunk, New Relic, AppDynamics, Compuware APM, Datadog, Oracle Management Cloud
The number of choices available for building a DevOps toolchain is vast, and new tools are added continually to help developers automate almost every aspect of the software development cycle. This enables a DevOps culture that can keep pace with today’s demands without sacrificing release quality.
The Importance of IaC in Application Development
When it comes to developing modern applications, there are different levels of abstraction that can be adopted. One of the most common methods is infrastructure abstraction. This can be obtained by using infrastructure as a service (IaaS), provisioning virtual machines (VMs), or containers in a cloud environment to deploy applications.
Today, many organizations and individual developers rely heavily on this level of abstraction for building application infrastructure.
Whether building VMs or Docker containers on a public cloud platform or deploying on-premises to a virtualized environment, this level of abstraction often involves a fair amount of manual configuration before or after provisioning.
IaC lets developers further accelerate application development by taking care of the heavy lifting of provisioning compute, networking, storage, security, identity, monitoring, logging, and other infrastructure.
The infrastructure on which applications run is automatically provisioned and managed from code, instead of manually.
Changes to the infrastructure are made by changing the code, and teams use source control management tools like Github to tie these changes to issues, goals, and people for accountability, and can be easily reverse changes if necessary.
Developers get a fast, repeatable process for deploying application infrastructure.
Changes to both the application and the infrastructure can happen quickly.
Hybrid cloud deployments may be better supported, if using cloud agnostic IaC engines
Service issues are detected and services are restored faster, since the infrastructure is managed by a DevOps team at the company.
DevOps culture is characterized by releasing applications faster and more frequently. When you combine this culture with the infrastructure automation features of IaC, you can further abstract and automate the backend heavy lifting required for application development.
To enable this higher level of abstraction, you take the principles that govern your software development process and apply them to your infrastructure using IaC tools like Terraform, Ansible, Chef, or Puppet. There are two types of IaC tools.
These tools enable the developer to specify the desired end state—that is, they are “outcome oriented.” If this desired state does not match the current state of infrastructure, the tool will perform a series of actions specified in configuration files. For example, Puppet and Terraform.
These tools enable the developer to specify a sequence of commands explicitly—that is, they are “action oriented.” For example, Chef.
IaC tools enable consistent, repeatable, and automated deployment of infrastructure, enabling you to focus on rapid application development, and not on waiting for infrastructure provisioning. When you adopt IaC, you can improve efficiency and reliability of your entire software development process by reducing manual, one-off infrastructure deployments or configurations.
Determining Whether to Move to IaC
Before making the move to IaC, it’s important to think about your own software development environment and determine if it’s the right fit. While IaC offers benefits in infrastructure automation, it may not be right for everyone.
Your current infrastructure is a large factor here. You could be leveraging public cloud IaaS entirely, some combination of public cloud IaaS and on-premises hardware, or on-premises hardware completely. Your application development environments may depend heavily on traditional virtualized infrastructure or physical servers, or may already be adopting container technologies like Docker or Kubernetes.
There may be a need to use serverless computing like Fn. Each of these approaches to handling application infrastructure will impact which level of abstraction will work best for your software development processes.
Beyond infrastructure considerations, there are business and cultural considerations as well. DevOps might be a new concept for some companies and developers, and not everyone is ready to adopt a DevOps culture, which includes IaC. However, with the accelerating pace of business, even companies that have not embraced DevOps are almost certainly feeling the pressure to speed up application development.
Other factors that will affect your choices around infrastructure abstraction are your current efficiency and speed of development, the control you have over your development environment, and your organization’s tolerance for risk.
How Oracle Cloud Enables DevOps and IaC
With Oracle Cloud, you can develop applications in a variety of languages, including Java, Python, Go, and PHP. You can build modern apps with the latest technologies using containers, microservices, serverless computing, and DevOps toolchains. You can also easily incorporate analytics, Internet of Things (IoT), mobile, and social into your apps. Development is open, modern, and easy.
Oracle gives you several tools to accelerate application development.
Oracle Developer Cloud Service for DevOps: Provides a hosted team development and delivery platform including issue tracking, Git code versioning, wiki, Agile development and planning tools, continuous integration and delivery automation with pipelines.
Oracle Cloud Infrastructure Resource Manager for IaC: A fully managed service for provisioning infrastructure resources using Terraform.
DevOps Tools and Plugins:
Oracle Cloud Platform Terraform provider: To interact with Oracle Cloud Infrastructure resources using Terraform.
Terraform modules: Self-contained packages of Terraform configurations that are managed together and used to create reusable components.
Ansible modules: To provide orchestration, provisioning, and configuration management, using Ansible, for Oracle Cloud Infrastructure services and resources (compute, load balancing, database, and so on).
Chef plugin: To provide an interface between Oracle Cloud Infrastructure and Chef server, thereby enabling configuration and management of nodes and applications.
Oracle Container Pipelines, Oracle Cloud Infrastructure Registry and Oracle Container Engine for Kubernetes for DevOps and containers: Services that empower development teams to quickly build, test, and deploy container-based applications.
Oracle Functions for serverless (in Limited Availability): Scalable, multitenant functions-as-a-service that lets you focus on writing code without having to worry about infrastructure concepts.
Find out more about other Oracle Cloud Native Services at: oracle.com/cloud-native
In addition, with Oracle Cloud you can bring your DevOps toolchain to us. This enables you to keep using the tools you’re using to deploy applications today, and we help you to deploy those apps quickly and easily.
Try it yourself
Take a free trial to start using Oracle Developer Cloud Service and Resource ManagerFree Cloud Trial
Navigating the Path to DevOps and IaC
Companies of all sizes are forced to develop products faster and provide optimal customer experience. They can achieve these goals through DevOps and IaC. Making the change to DevOps is one of the biggest challenges software development teams face today. The move is much more than just a technological or process change—moving to DevOps and IaC requires a shift in the way developers and infrastructure operators work together to create, test, and deliver software continuously.
For many companies, development and operations don’t just have different roles but different priorities as well. A successful transition to DevOps and IaC means unifying these groups, getting them on the same page and working
towards the same goals. This cultural change is much larger than the technology changes that come with DevOps.
Despite challenges, adopting a DevOps culture and IaC tools to automate the heavy lifting of provisioning and managing of the DevOps toolchain can provide great benefits to the company. These include improved efficiency in development, better consistency in app deployment, more stable/reliable applications, and more satisfied users.