Moving away from Travis CI

At rOpenSci, we encourage R package developers to take advantage of Continuous Integration services to automatically check the package on different platforms, with different versions of R. The rOpenSci dev guide dedicates chapter 2 to the topic of Continuous Integration Best Practices, and recommends a few common CI vendors, including Travis CI.

Travis CI has been a pioneer in free public CI services, and made the concept popular in the open-source community. The service started to get wide adoption in 2012, and native support for R was added by Craig Citro in 2015, with more contributions from current maintainers Jim Hester and later Jeroen. In 2016 we wrote a blog post about using a build-matrix in order to check your packages on multiple versions of Linux and MacOS, which is super powerful for R package development.

A Change of Management

Sadly, good times came to an end. In 2019 the company was sold to a private equity firm, and soon after the acquisition, a large portion of the senior engineer staff was layed off. Under the new management, open-source users started suffering from significant outages and backlogs, while being pushed towards the new enterprise product travis-ci.com (the original Travis service was hosted on travis-ci.org).

The big blow came earlier this month, when Travis announced a new pricing model which no longer has a generous free tier for open-source projects, and also fully shutting down the old travis-ci.org product by December 31. It is still unclear what exactly the new product will look like, perhaps it can still be useful, but with the direction the company is heading, we recommend exploring other options.

GitHub Actions

Fortunately, there are many other companies offering free CI for open-source projects these days. Some popular vendors include AppVeyor, Circle-CI, Azure Pipelines, and Gitlab CI, but the biggest new player is GitHub Actions: the native CI/CD system from GitHub.

The GitHub Actions (GHA) platform was introduced only recently, but has quickly taken over the open-source world. The system is extremely flexible, allowing you to run any combination of scripts, containers, and imported actions from other users. The native integration with GitHub this takes away the annoying authentication dance that is required for third party services, making the setup completely seamless. Very generous free build resources are provided for open-source projects, but if you need something else, GHA also allows you to plug in self-hosted runners, giving you complete control of your hardware and build environment.

To get started with GHA for R, the r-lib/actions repo has a number of preconfigured actions and example configurations written by Jim Hester (again), for installing R, running checks, etc. Simply copy the check-standard.yaml file into the .github/workflows/ folder of your R package ¹, then push, and see the magic happen. Note that Jim’s scripts are only one example: GHA will let you run any script in the OS or container your choice, allowing you to fully customize what happens on each new push, pull request, opened issue, etc.

Thank you Travis !

We will likely only start seeing more of GitHub actions, as it becomes the default CI for open-source. But for now I would like to thank the Travis team, especially the initial founders and engineers, for bringing massive free CI to the open-source ecosystem.

I remember the first time installing Travis on a project, and seeing commits and pull requests automatically get built and checked, without having to do anything. I was blown away, it really brought the concept of pull requests to life. In no time, CI became an integral part of development, providing an efficient workflow to test new features and review pull requests, and green badges started appearing everywhere. I think it is fair to say that we could never maintain the number of projects we do today, if we had to test all those pull requests manually!

The R community certainly is not alone in this. A user in the hackernews discussion comments:

Free CI really did provide a massive boost to collaborative open-source projects. As a user, it did a lot to increase software quality: not just catching inadvertent bugs, but also ensuring that there was at least some reproducible way to get the code working, that didn’t depend on some implicit configuration of the authors system. As a maintainer, accepting simple pull requests becomes much easier when you can quickly look over the code and check the CI status, and not have to try it out locally yourself. It was certainly critical to the “social coding” idea behind GitHub.

Of course the open-source community is sad to see Travis become an enterprise-first product. But in all honesty, the system is no longer state-of-the-art, and probably won’t be able to compete with the new GitHub/Microsoft products.

Also we are very aware that it is difficult to find a sustainable business model around open-source, and hope the company will survive by refocusing on specialized enterprise CI needs. Nevertheless, as we transition towards a new generation of CI systems, we won’t forget the pioneering role that Travis played in taking open-source collaboration to the next level. Thank you!

Migrating your Projects

If you are still using Travis-CI, there are several options. First, you should convert your travis-ci.org account into a travis-ci.com account following instructions, if you had not done so already. This will make sure your CI keeps working after December 31, at least until you run out of credit :-)

As mentioned, we recommend exploring alternatives, in particular GitHub actions. As things are evolving rapidly, you can keep an eye on the r-lib/actions repository, our dev guide, and of course have a look at what other package authors are doing.

Fortunately the vendor lock-in with CI is pretty limited. For standard configurations you can just replace the .travis file with an equivalent template from another CI service. For customized configuration, you may have to spend a few minutes translating that to another format, which is the price we pay for using these cloud services. Feel free to reach out to the rOpenSci community if you could use some help!