Why You Should Use Continuous Integration and Continuous Deployment

Having a bad development workflow will cost you a lot. It can make your senior engineers be productive like juniors, or even worse push them to leave the company. A great workflow can make push your good developers to be great and your best to be exceptional.

Getting this workflow in order is one of the most critical steps any business and team can take.

Two best practices that have gained a lot of traction over the last years are Continuous Integration and Continuous Deployment.

Continuous Integration is the practice of testing each change done to your codebase automatically and as early as possible. But this paves the way for the more important process: Continuous Deployment.

Continuous Deployment follows your tests to push your changes to either a staging or production system. This makes sure a version of your code is always accessible.

First, let us have a look at Continuous Integration. I’ll show you how to get started with testing. Later we will talk about Continuous Deployment which is the next logical step and will increase efficiency of your development team tremendously.

Continuous Integration

Every step in your development workflow that can be automated should be. You need this automation to be able to focus all your time on building your product and providing value to your customers. Testing is one of those steps where automation is key.

Automated testing allows you to implement new features quickly, as you can always prove that your most important features still work as you expect. This gives you the confidence to experiment.

How to start testing software

When starting testing your software it’s important to view the app from a user’s perspective. Let’s have a look:

  • What are the typical workflows?
  • What features do users go through often?
  • What steps do users go through often?

Get started with Selenium, PhantomJS or other frontend testing tools to make sure your application works as expected from the users perspective. Move from testing the user interface to testing lower level code, not the other way around.

Often teams struggle with defining those first important test cases. We’ve come up with a guide on how to start testing, so take a look and see if this works for your team as well.

Tools you can use to start testing your frontend:

Now that we have a strong base of tests let’s go into lower levels and start with Unit-Testing. We want to make sure that the workflows are fine and that our UI works. With every change!

You have the tools, you know how to do it. Now the most important thing is: Get started with testing! It pays off immediately, not just from a software development perspective. You can read about the business advantages of Continuous Integration in this blog post by Joe Green.

Continuously running your tests

It’s too easy for us as developers to say that a specific change won’t break something and later realise it did break the app. Often we are simply too lazy to really run all of our tests. This is where an automated system that runs or tests whenever we do any changes continuously can help us from falling into the trap.

Oftentimes teams work on 2 week or even longer sprints. At the beginning of the sprint a developer pushes some changes that breaks a test. The team doesn’t have continuous integration, so nobody detects the error for a week. At the end of the sprint, before deploying to production, the lead developer runs all tests on a test machine to make sure everything is fine. And only at this point does the team detect the failed test. Now going back and determining which change broke the test is hard and takes a long time, totally unnecessarily. The failing test could have been detected only minutes after it was pushed, but as there was no automated system in place the team wasted precious time late in the sprint, instead of fixing it early on.

The longer the sprint, the more costly this cycle becomes. This workflow doesn’t help the team, the single developer and especially not the company. Relying on automation to tell us early on when something goes wrong needs to be at the core of your development workflow.

Continuous Integration is the first, but very important step in Continuous Deployment.

Have a look at Martin Fowler’s in-depth article about Continuous Integration to learn more about it.

Continuous Deployment

Code rots. It should always run somewhere. There are a lot of changes that can break your app without you realizing it. Think about external dependencies being updated, API’s changed, server packages installed… The list goes on. Often we do not have the ability to control every part of our infrastructure. We rely on other products and companies to maintain parts of our stack.

This is where Continuous Deployment shows all of it’s power. Whenever your main code branch passes the tests it should immediately be deployed to at least a staging environment. Even better if you can go to production immediately. With the master branch running at least on staging you can always have your QA or development team take a look or even use this last version. This shortens the cycle until you find an error or usability problem tremendously.
It moves finding those problems very early into the lifecycle, where it is cheap to fix them, instead of late where you often can’t fix them any more.

Configuring a deployment pipeline

Deployment pipelines describe how a change in your application moves through your infrastructure into production. It lays out all necessary automated and manual steps. Every step that can be automated should be, to make the deployments as productive as possible.

Having this predefined workflow pipeline in place will make sure you follow all the necessary procedures when you have to ship a fix quickly. A small mistake could mean a downtime, so relying on automation is crucial for this.

The ability to ship, get feedback from your customers and iterate on your product quickly is a major competitive advantage. Intercom wrote a blog about why shipping is your companies heartbeat. Especially when building the first iterations of your product the importance of shipping quickly and regularly can’t be overstated.

Today, as the costs for creating and publishing software shrunk dramatically, it is vitally important to get your product to your customers quickly. Your competition has the same cheap and productive tools available. Getting the most out of your workflow and push your productivity to the next level is how you can differentiate. Continuous deployment can help you make this happen.

Getting started

We’ve identified several steps that can help you get started with continuous deployment. These steps have come out of interviews with dozens of companies and constant feedback from the thousands of developers using our product.

Test everything, but in a smart way

Automation is the key to pushing your productivity. Testing a vital step to automate all your quality control. Whenever you want to change your product you need to be able to quickly and automatically assess that everything still works. Getting to this point assures you can iterate and innovate quickly without having to worry about breaking things. It allows to push much farther than you were ever able before.

Tests are first and foremost a tool to keep your users happy. You don’t put effort into testing to make your development team happy or fulfill some vague workflow, but to be sure that you don’t break stuff your customers use. And additionally also to be able to implement the next features they want and pay you for. So start testing with the user in mind. Start testing the Frontend and interactions there first, before moving to lower level tests. Remember we test to make our users happy, not just to create better software.

Tests make software as Clemens laid out in his blog post.

Example

This script tests that when entering good day into google translate it is properly translated from german into english.


With CasperJS you can start testing your front-end quickly and move to more complex scenarios later.

Automate Deployment

Every step in your deployment has to be automated. There can’t be any manual steps involved, as you will fail to do them once, which will kill your application. As you are aware you automatically start deploying less often. This creates a vicious cycle where the less you deploy the worse your deployment infrastructure becomes. Continuous Deployment works because of automation, so make sure you invest enough time here.

Automate Rollback

Automating rollbacks is critical when pushing regularly. Otherwise you will always fear to push more. Rollback includes your application code but also your database. Regular backups are an absolute must. Try to restore them regularly as well, otherwise you will never be sure if it really works until you desperately need them.

Deploy to Staging

Automatically push your master branch into a staging application. The staging application is your first line of defense, so you can easily review any changes going into production. This can even include automated tests, so you know your staging server works fine.

Use your staging environment

Using your own product is incredibly helpful to determine problems in your application. While using your production server is fine, using your own staging system is even better. It makes sure that you immediately detect errors that might take down your application, as you team always uses the latest available codebase.

Automatically deploy to production

Finally deploy continuously to your production environment. This workflow will dramatically change the way your whole organization thinks about releasing software. It is not just a little faster deployment, but needs to be planned and included in your daily work and planning. It’s been an important part in the innovation of some of the largest tech companies in the world and you should give it a trie too.

Conclusions

Finding new ways to become more productive is important for every software team. Getting slower, especially in comparison to your competition, could easily harm or even kill your company and product. You want to be able to push quickly and regularly to your customers, so they get the best product possible. Continuous Deployment can get you a long way there.

If any questions come up or if there is anything else we can help let us know either here in the comments or send us a tweet to @codeship. We’ve also recently released a free ebook that walks you through our development workflows and can help you dive deeper into Continuous Deployment. You can grab the book here.

Ship long and prosper!

Relevant links and further information

Free Workshops

Watch one of our expert, full-length teaching videos. Choose from either HTML, CSS or Wordpress.

Start learning

Florian Motlik

Florian is CTO of Codeship, a hosted Continuous Integration and Continuous Deployment platform. Codeship provides the simplest way to get started with Continuous Integration and Deployment. You can set up your project in a few minutes and let Codeship run your test suites and automatically deploy on a successful build. Codeship hosts all of the infrastructure and takes care of scalability and integrations to third party providers, so you can focus on building your product. All major languages, databases, datastores and tools are supported. If you are interested you can try Codeship for Continuous Integration and Continuous Deployment here.

Comments

11 comments on “Why You Should Use Continuous Integration and Continuous Deployment

  1. Greate article, on the spot.
    ps: Please, fix the broken/wrong link in “how to start testing”.

  2. Good blog post. We are using already continuous integration in the cloud. Continuous deployment makes totally sense! That’s the next thing I will introduce at VersionEye.

  3. Hi Florian,

    This post focuses on the advantages of CT and CI and it doesn’t mention any of their disadvantages such as:

    – Some bugs are not caught with CT.
    – The learning curve is not as easy as one thinks it is for these systems.
    – The make developers less careful about their code.

    • Thanks for your feedback. Some of my thoughts to your different points:

      >Some bugs are not caught with CT
      CI is mostly about not killing a feature that was implemented in the past. It’s not so much about finding new bugs, but not redoing old ones. When pushing larger features testing them on staging or QA first and doing a hands on session where you just try to break the feature is very helpful. All those can then be put into automated tests to make sure it works as expected after any change in the future.

      >The learning curve is not as easy as one thinks it is for these systems
      I’m obviously biased, but as one of the founders of a company who provides a hosted solution for running your CI infrastructure (https://www.codeship.io) I think it is very easy now to get started and understand the workflow. With this and many other blog posts we are trying to educate people as well, so it becomes a non-issue.

      >The make developers less careful about their code
      CI can’t be the only step to guarantee their code works well. At Codeship for example we do code reviews for every piece of code we release into production. This is another step to guarantee the quality of our code. And from personal experience I don’t think having tests in place makes people put less care into the quality of their code. I honestly don’t think there is an alternative. Not testing is just not an option if you want to provide a successful and great product over longer term. Manual testing is too brittle and bugs can creep in way too easily. Manual/Exploration testing has it’s place, but can’t guarantee the long term stability of your product. I personally don’t think it works at all, but it’s definitely not very efficient in either case.

      What CI does is make sure that you are aware of all the influence a specific changes has on the rest of the system. Something you might not have thought about can be easily caught and fixed by a proper test. And this in turn makes our customers happy, which makes us money.

  4. How do you handle continuos integration where the requesting business units tend to be indecisive about what they want deployed to Production? CI seems perfect if you are always promoting the latest build to staging, but what happens when a feature gets built and while it meets the requirement technically, the idea turns out to be better than the finished product? The problem becomes that during QA is when this comes out, but by that time development has already begun on the next sprint.

    What is common practice for a situation like this?

    Great article. It’s rare when an article hits so close to home.

    • >CI seems perfect if you are always promoting the latest build to staging, but what happens when a feature gets built and while it meets the requirement technically, the idea turns out to be better than the finished product?

      There can still be a QA process wher you can decide on what to deploy. The most important thing is to automate very step in the whole chain, so deploying becomes a non-issue and you can start doing it all the time. From this you can build the workflow that is most suitable for your organisation. Several steps you can take to test features:

      *) Deploy to a staging server continuously so constant feedback on features can be done
      *) Deploy with feature flags so you can easily turn off a feature
      *) Deploy a minima viable feature, so build the smallest thing possible to test if that feature is interesting to customers. If it works, build on top of that.

      That’s at least some of the strategies I’ve seen.

  5. Really good article and i agree on the implementation of CI. But there should also be a good review system in palace as making sure the test pass is not enough we need to ensure that quality of code is also maintained.