Having a bad development workflow will cost you a lot. It can make your senior engineers be productive like juniors, or even worse push them to leave the company. A great workflow can make push your good developers to be great and your best to be exceptional.
Getting this workflow in order is one of the most critical steps any business and team can take.
Two best practices that have gained a lot of traction over the last years are Continuous Integration and Continuous Deployment.
Continuous Integration is the practice of testing each change done to your codebase automatically and as early as possible. But this paves the way for the more important process: Continuous Deployment.
Continuous Deployment follows your tests to push your changes to either a staging or production system. This makes sure a version of your code is always accessible.
First, let us have a look at Continuous Integration. I’ll show you how to get started with testing. Later we will talk about Continuous Deployment which is the next logical step and will increase efficiency of your development team tremendously.
Every step in your development workflow that can be automated should be. You need this automation to be able to focus all your time on building your product and providing value to your customers. Testing is one of those steps where automation is key.
Automated testing allows you to implement new features quickly, as you can always prove that your most important features still work as you expect. This gives you the confidence to experiment.
How to start testing software
When starting testing your software it’s important to view the app from a user’s perspective. Let’s have a look:
- What are the typical workflows?
- What features do users go through often?
- What steps do users go through often?
Get started with Selenium, PhantomJS or other frontend testing tools to make sure your application works as expected from the users perspective. Move from testing the user interface to testing lower level code, not the other way around.
Often teams struggle with defining those first important test cases. We’ve come up with a guide on how to start testing, so take a look and see if this works for your team as well.
Tools you can use to start testing your frontend:
- Selenium (Frontend Testing)
- CasperJS (Frontend Testing)
- Cucumber or Lettuce (Behaviour Driven Testing for Ruby and Python)
Now that we have a strong base of tests let’s go into lower levels and start with Unit-Testing. We want to make sure that the workflows are fine and that our UI works. With every change!
You have the tools, you know how to do it. Now the most important thing is: Get started with testing! It pays off immediately, not just from a software development perspective. You can read about the business advantages of Continuous Integration in this blog post by Joe Green.
Continuously running your tests
It’s too easy for us as developers to say that a specific change won’t break something and later realise it did break the app. Often we are simply too lazy to really run all of our tests. This is where an automated system that runs or tests whenever we do any changes continuously can help us from falling into the trap.
Oftentimes teams work on 2 week or even longer sprints. At the beginning of the sprint a developer pushes some changes that breaks a test. The team doesn’t have continuous integration, so nobody detects the error for a week. At the end of the sprint, before deploying to production, the lead developer runs all tests on a test machine to make sure everything is fine. And only at this point does the team detect the failed test. Now going back and determining which change broke the test is hard and takes a long time, totally unnecessarily. The failing test could have been detected only minutes after it was pushed, but as there was no automated system in place the team wasted precious time late in the sprint, instead of fixing it early on.
The longer the sprint, the more costly this cycle becomes. This workflow doesn’t help the team, the single developer and especially not the company. Relying on automation to tell us early on when something goes wrong needs to be at the core of your development workflow.
Continuous Integration is the first, but very important step in Continuous Deployment.
Have a look at Martin Fowler’s in-depth article about Continuous Integration to learn more about it.
Code rots. It should always run somewhere. There are a lot of changes that can break your app without you realizing it. Think about external dependencies being updated, API’s changed, server packages installed… The list goes on. Often we do not have the ability to control every part of our infrastructure. We rely on other products and companies to maintain parts of our stack.
This is where Continuous Deployment shows all of it’s power. Whenever your main code branch passes the tests it should immediately be deployed to at least a staging environment. Even better if you can go to production immediately. With the master branch running at least on staging you can always have your QA or development team take a look or even use this last version. This shortens the cycle until you find an error or usability problem tremendously.
It moves finding those problems very early into the lifecycle, where it is cheap to fix them, instead of late where you often can’t fix them any more.
Configuring a deployment pipeline
Deployment pipelines describe how a change in your application moves through your infrastructure into production. It lays out all necessary automated and manual steps. Every step that can be automated should be, to make the deployments as productive as possible.
Having this predefined workflow pipeline in place will make sure you follow all the necessary procedures when you have to ship a fix quickly. A small mistake could mean a downtime, so relying on automation is crucial for this.
The ability to ship, get feedback from your customers and iterate on your product quickly is a major competitive advantage. Intercom wrote a blog about why shipping is your companies heartbeat. Especially when building the first iterations of your product the importance of shipping quickly and regularly can’t be overstated.
Today, as the costs for creating and publishing software shrunk dramatically, it is vitally important to get your product to your customers quickly. Your competition has the same cheap and productive tools available. Getting the most out of your workflow and push your productivity to the next level is how you can differentiate. Continuous deployment can help you make this happen.
We’ve identified several steps that can help you get started with continuous deployment. These steps have come out of interviews with dozens of companies and constant feedback from the thousands of developers using our product.
Test everything, but in a smart way
Automation is the key to pushing your productivity. Testing a vital step to automate all your quality control. Whenever you want to change your product you need to be able to quickly and automatically assess that everything still works. Getting to this point assures you can iterate and innovate quickly without having to worry about breaking things. It allows to push much farther than you were ever able before.
Tests are first and foremost a tool to keep your users happy. You don’t put effort into testing to make your development team happy or fulfill some vague workflow, but to be sure that you don’t break stuff your customers use. And additionally also to be able to implement the next features they want and pay you for. So start testing with the user in mind. Start testing the Frontend and interactions there first, before moving to lower level tests. Remember we test to make our users happy, not just to create better software.
Tests make software as Clemens laid out in his blog post.
This script tests that when entering good day into google translate it is properly translated from german into english.
With CasperJS you can start testing your front-end quickly and move to more complex scenarios later.
Every step in your deployment has to be automated. There can’t be any manual steps involved, as you will fail to do them once, which will kill your application. As you are aware you automatically start deploying less often. This creates a vicious cycle where the less you deploy the worse your deployment infrastructure becomes. Continuous Deployment works because of automation, so make sure you invest enough time here.
Automating rollbacks is critical when pushing regularly. Otherwise you will always fear to push more. Rollback includes your application code but also your database. Regular backups are an absolute must. Try to restore them regularly as well, otherwise you will never be sure if it really works until you desperately need them.
Deploy to Staging
Automatically push your master branch into a staging application. The staging application is your first line of defense, so you can easily review any changes going into production. This can even include automated tests, so you know your staging server works fine.
Use your staging environment
Using your own product is incredibly helpful to determine problems in your application. While using your production server is fine, using your own staging system is even better. It makes sure that you immediately detect errors that might take down your application, as you team always uses the latest available codebase.
Automatically deploy to production
Finally deploy continuously to your production environment. This workflow will dramatically change the way your whole organization thinks about releasing software. It is not just a little faster deployment, but needs to be planned and included in your daily work and planning. It’s been an important part in the innovation of some of the largest tech companies in the world and you should give it a trie too.
Finding new ways to become more productive is important for every software team. Getting slower, especially in comparison to your competition, could easily harm or even kill your company and product. You want to be able to push quickly and regularly to your customers, so they get the best product possible. Continuous Deployment can get you a long way there.
If any questions come up or if there is anything else we can help let us know either here in the comments or send us a tweet to @codeship. We’ve also recently released a free ebook that walks you through our development workflows and can help you dive deeper into Continuous Deployment. You can grab the book here.
Ship long and prosper!
Relevant links and further information
- Introduction to Continuous Integration by Martin Fowler
- Why Continuous Deployment by Eric Ries
- Continuous Delivery by Jez Humble
- A Business Case for Continuous Integration
- How To Start With Testing From Top To Bottom
- Start Testing Your Website With CasperJS
- The Codeship: A hosted Continuous Integration and Continuous Deployment Platform
- The Codeship Blog about Continuous Deployment, Continuous Integration and Software Testing
- Codeship’s Free Ebook: Efficiency in Development Workflows