DevOps
Steven J Zeil:
Last modified: Dec 27, 2023
Contents:
Abstract
DevOps is a merger of Development and Operations.
It can variously be viewed as a reaction to or an extension of Agile development.
1 Motivation
Tension between Incremental Development and Traditional Operations
- Incremental development promises the availability of multiple useful versions of the software.
- Agile development suggests that these increments will be available with only short time intervals (on the order of a few weeks) between them.
- This multiples the amount of work required to systems test and deploy software
- Tasks traditionally performed by a group separate (“siloed”) from the developers.
(figure from Atlassian)
DevOps attempts to merge development
- plan
- build
- continuous integration
with operations
- continuous deployment
- monitor
- operate
2 Challenges
- Developers often unaware of QA issues and consequent operational roadblocks.
- Lack of feedback on utility & value of implemented features.
- Unclear delineation of responsibilities for problems encountered during QA or operation
2.1 Values
- Collaboration & communication
- Agile, incremental methods
- Automation
- Version control
- CI Servers
- Virtualization
- Automated system tests
- Infrastructure as Code
- Monitoring
3 Infrastructure as Code
Manage infrastructure (servers, load balancers, network) via configuration files that allows
- Configuration files describe a model of servers, network connections, load balancers, etc.,
- Configuration management tools interpret that model and “build” the infrastructure accordingly.
How can you “build” infrastructure automatically? A mixture of
- selecting & connecting to existing hardware
- using virtualization to simulate new hardware
3.1 Puppet
- Linux tool for automating config mgmt
- Master-client architecture
- Provides a declarative language, PuppetDSL, for describing desired configuration
case $operatingsystem {
centos, redhat: { $service_name = 'ntpd' }
debian, ubuntu: { $service_name = 'ntp' }
}
package { 'ntp':
ensure => installed,
}
service { 'ntp':
name => $service_name,
ensure => running,
enable => true,
subscribe => File['ntp.conf'],
}
file { 'ntp.conf':
path => '/etc/ntp.conf',
ensure => file,
require => Package['ntp'],
source => "puppet:///modules/ntp/ntp.conf",
# This source file would be located on the Puppet master at
# /etc/puppetlabs/code/modules/ntp/files/ntp.conf
}
3.2 Chef
Systems administration tasks organized into “cookbooks” and “recipes”.
- Linux, Windows, cloud-friendly (AWS)
- Configurations are procedural in Ruby DSL
- Master-client
config.vm.box = CFG_BOX
config.vm.hostname = CFG_HOSTNAME
config.vm.provider "virtualbox" do |v|
v.name = 'chef-automate'
v.memory = 8192
v.cpus = 4
v.customize ['modifyvm', :id, '--audio', 'none']
end
config.vm.synced_folder '.', '/opt/a2-testing', create: true
config.vm.network 'private_network', ip: CFG_IP
config.vm.provision 'shell', env: {'CFG_IP' => CFG_IP,
'CFG_HOSTNAME' => CFG_HOSTNAME,
'ACCEPT_CHEF_TERMS_AND_MLSA' => AcceptLicense.new}, inline: $deployscript
end`
3.3 Ansible
- Written in Python, configuration based on YAML scripts
- Still heavily procedural(?)
- Single-node architecture
- Uses ssh to build artifact on target systems
3.4 Terraform
- Focus on provisioning, setting up servers before building the configuration
- Reputedly easier to use with Docker
- More declarative approach
4 Pipelines
Automation of continuous deployment is usually organized into pipelines.
- An assembly line of smaller stages.
- The “pipeline” generally refers to both the tools and the collection of steps automated using those tools.
- Typically include CI, automated (system) testing, report generation & continuous feedback (monitoring)
- Does not necessarily map to “pipelines” as provided by GitHub & GitLab
- Problem of passing artifacts from one stage to the next in a pipeline
Contrasting visions of pipelines
5 Monitoring
- Identify appropriate metrics
- Continuously collected
- Real-time analytics & reporting
Identifies both operational problems and provides feedback to Agile planning.
5.1 Elastic-Logstash-Kibana (ELK)
- E: search & analytics engine
- L: server-side logging
- K: visualization
Provide for capture & visualization of properties such as
- log files
- cloud data
- availability
- network traffic