Awareness of costs to downstream users

Most interviewees stated that they avoid breaking changes whenever possible. Almost all interviewees stated that they started with a presumption not to make changes that would affect downstream users, when they could avoid it. Reasons included looking out for their users' best interests and knowing that costs to affected users would come back to them, as users ask for help adapting to the change, ask for the change to be reverted, or seek alternative packages. Two interviewees specifically mentioned concern for downstream users' research ( We're improving the method, but results might change, so that's also worrying - it makes it hard to do reproducible research).

Our interviewees' concern for impacts on users was amplified by the size and visibility of the user base, and the perceived importance and appropriateness of their usage. In fact, many interviewees across all ecosystems had an awareness of their user community and were concerned specifically about the number of users affected and the quantity of complaints that a change would imply, e.g., Sometimes you want to rename a function or class, ... However because this would break scripts or packages assuming the old name, you often end up supporting both names. Some interviewees noted that their sensitivity toward avoiding breaking changes grew with experience and with a growing user base, as they learned from feedback received about earlier breaking changes.

Only a few developers were not particularly worried about breaking changes. Some had a strong tie to their users and felt they could help them individually ( We try to avoid breaking their code - but it's easy to update their code). One interviewee expressed an "out of sight, out of mind" attitude: Unfortunately, if someone suffers and then silently does not know how to reach me or contact me or something, yeah that's bad but that suffering person is sort of [the tree] in the woods that falls and doesn't make a sound as far as I'm concerned.

Finally, there was some debate about bug-fixing changes (see above). While some developers aimed to support downstream users who relied on incorrect behavior, other developers were less concerned when they considered usage as inappropriate, as stated for example: After upgrading the parser some people complained that their script was no longer working. But the problem was that their syntax was invalid to begin with. It's obviously their fault.

Techniques to Mitigate or Delay Costs

Despite a strong general preference for avoiding breaking changes, there are many cases where the opportunity costs of not making a change are too high. Our interviewees identified several different strategies for how they, as package maintainers, routinely invest effort to reduce or delay the impact from their changes for downstream users.

Maintaining old interfaces

Across all ecosystems, preserving the old interface alongside a new one is a very common approach to mitigate an immediate impact of a change on downstream users. While specifics depend on the language and tools, documenting methods as deprecated and providing default implementations for new extension points or parameters are common strategies to avoid breaking downstream implementations. In all these cases, the package developer invests additional effort now to preserve backward compatibility and accepts technical debt in the package for maintaining the extra code until it is eventually removed, in exchange for preventing an immediate impact of the change. The developer may at some later time clean up the code, affecting downstream users that have not updated in the meantime.

Similarly, many interviewees told us about various techniques to perform changes without breaking binary compatibility. They prevent rework costs for existing users by accepting more complicated implementations and harder maintenance in the changed package, while possibly also creating costs for new downstream users who have to deal with more complicated mechanisms.

Parallel Releases

Several developers reported strategies to maintain multiple parallel releases, such that downstream developers can delay updating or skip releases with larger breaking changes, while still incorporating minor nonbreaking changes (e.g., bug fixes) from a separate release. Node.js/npm offers specific mechanisms to support parallel releases with different version numbers; it is a common practice to provide security patches also for older releases. In contrast, CRAN only supports sequential version numbering, causing some developers to fork their own packages (e.g., `reshape' to `reshape2'). In each case, developers invest significant additional effort in maintaining old releases to reduce the (immediate) impact on downstream users.

A variant of this strategy is to maintain separate interfaces for different user groups with different stability commitments within the same package (see the façade pattern). For example, one Eclipse interviewee provided in parallel both a detailed and frequently changing API for expert users and a simpler and stable API that insulated less sophisticated users from most changes. Similarly, an R interviewee has split packages into smaller packages, with the intention that each user could depend only on parts relevant to them and would be exposed to less change. In both cases, the developer accepts the higher design and maintenance costs of multiple APIs for reduced impact on specific groups of users with distinct needs.

Release Planning

Some individual developers and some communities are considerate of downstream users when planning when to release changes. Some developers report deliberately delaying changes to batch multiple changes together. For example, an R interviewee keeps versions of his package with a quickly-changing API in a separate repository and updates CRAN less frequently when he wants to release a version to a broader audience. While in R/CRAN and Node.js/npm packages are usually released independently, large parts of the Eclipse community coordinate around synchronized yearly releases (a strategy also common in other package systems as Debian and Bioconductor). Delaying releases may incur coordination overhead and opportunity costs in slowing down development for the changer, but reduces the frequency (though not necessarily the severity) with which downstream users are exposed to changes and gives downstream users a planning horizon.

Communication with users

Finally, developers use various forms of communication with downstream users to reduce the impact of a breaking change. Several interviewees made early announcements to create awareness and receive feedback. For example, one interviewee explained that two weeks or a month before the actual release, I do sort of a pre-release announcement on Twitter [and] tell people to use the README. Since our interview he has written a script to email all downstream maintainers before a release.

Another reason for communicating with downstream users was to help them deal with the aftermath of change. In the simplest case, a developer could invest effort in documenting how to upgrade. Several interviewees, were aware of their users and reached out to them individually; for example on interviewee contacted users using an old API to help them migrate, and another had most users present on-site and could therefore help them migrate their code. Another went so far as to create individual patches for downstream packages to get them to adopt a new interface and move away from an old deprecated one. In all cases, package maintainers invest effort to reduce costs for downstream users.

Coping with upstream change

Just as package developers have some flexibility in planning changes that may affect downstream users, developers have flexibilities regarding whether, when, and how to react to upstream change, again influenced by values, policies, and technologies. Having to monitor and react to upstream change can be a significant burden on developers (e.g., mismatch between schedules has been shown to be a barrier to collaboration). The urgency of reacting to change can depend significantly on the development context and platform mechanisms.

When discussing how frequently they react to upstream change, our interviewees described a spectrum ranging from never updating to closely monitoring all changes in upstream packages. Some interviewees explicitly ignored certain upstream changes; others upgraded dependencies only during their own releases or during deliberate house-cleaning sweeps. Even when the platform does not require updates, developers often prefer to update their dependencies to incorporate new fixes and features or to avoid accumulating technical debt. But they avoid updating when updates require too much effort (e.g., by causing complicated conflicts) or cause too much disruption downstream.

Monitoring Change

When developers have to or want to react in a timely fashion to upstream changes, they need to monitor the upstream projects in some way. The platform itself, e.g., Node.js, R core, and the CRAN infrastructure, is often an additional source of changes that developers need to keep up with. In our interviews, we discovered many different strategies for monitoring, including technical and social strategies. Their strategies varied along with the urgency of their needs, from active monitoring of upstream activity, to general social awareness of upstream activities, to a purely reactive stance where developers wait for some kind of notifications.

Burden of active monitoring

Only a few interviewees reported actively monitoring upstream changes by regularly looking at all changes of their upstream dependencies. Only three said they used GitHub's notification feed with some regularity. Several interviewees indicated that such raw notification feeds, in their current form, are a significant burden with a low signal to noise ratio, as stated for example by The quantity of notifications I get on GitHub already is to the point of overwhelming. So I don't even mostly read them unless I'm actually working on the project at that moment. He later told us that after our interview he tried scaling back to watching just the 3-5 projects he is actively working on. Only one interviewee did not feel overwhelmed, saying that occasional, casual skimming of GitHub feeds was useful way to get a casual overview of activity.

In several cases, developers monitored upstream changes not as outsiders following a stream of data, but as active participants in those projects, collaborating to influence them toward their own needs or providing direct contributions to those packages.

Others actively compiled and tested their project with development versions of upstream dependencies, emphasizing the importance of giving timely reactions: if you report it within a week there's a better chance the developer might remember what they did [...] which provides a good chance that they can revert their change before they hit their milestone.

Broad awareness through social mechanisms

Many interviewees tried to maintain a broad awareness of change through various social means. The most frequently mentioned mechanism, especially in the Node.js community, was Twitter. For example, one interviewee commented the people who write the actual software are fairly well connected on Twitter, [...] like water cooler type of thing. So we tend to know what's going on elsewhere. In each ecosystem, several interviewees mentioned the importance of face-to-face interactions at conferences for awareness about important changes in the ecosystem. Other mentioned social mechanisms to learn about change were personal networks, blogs, and curated mailing lists. Though these mechanisms are rarely specific to individual packages, several developers mentioned them as their main monitoring strategies.

Reactive monitoring

Although our research questions led us to probe interviewees about the aforementioned active and social monitoring practices, in fact most interviewed developers adopted a reactive strategy for most of their dependencies. They wait to hear about problems from others (in advance, or after things had broken): upstream developers contacting them about breaking changes, failing tests after dependency updates, or platform maintainers warning of changes that would affect them. There are also tools that enable this reactive stance, by creating targeted notifications on certain kinds of changes. The specific tools differ among the platforms and support different practices or policies. Policies and common practices (e.g., testing practices) in the platform strongly affect the reliability of a reactive strategy and corresponding tools.

Some Eclipse and Node.js/npm developers use continuous integration to detect compile-time issues caused by breaking changes in upstream packages early. For Node.js/npm, some developers use the tools gemnasium and greenkeeper to get notifications about new releases of upstream packages. Gemnasium alerts developers of package releases that fix known vulnerabilities, whereas greenkeeper submits pull requests to automate a continuous integration run against the new release. In either case, developers can react to notifications by email or pull requests.

CRAN is interesting because, by asking upstream developers to notify their users, it encourages downstream developers across the ecosystem to take a reactive stance (in contrast to Eclipse and Node.js/npm, where individual downstream developers need to employ optional monitoring tools). Some interviewees defended the practice of waiting to be told about breaking changes as a principled attention-preserving choice, consistent with ecosystem norms; others, however, were apologetic about being reactive: I guess I'll sound crass about this and say it. For things like that I would wait to hear from CRAN when something broke. Because I don't think I can keep up with all of it. CRAN enforces this policy with manual and automated checking on each package update, running the package's tests and the test of all downstream packages in the repository, as well as some static checks. The CRAN team may then warn an affected downstream developer of an upcoming change by email.

Reducing the Exposure to Change

Many developers have developed strategies to reduce their exposure to change from upstream modules and, thus, reduce their monitoring and rework efforts. The degree to which developers adopt such mitigation strategies again depends on the technology, policies, and values, as we will discuss.

Limiting dependencies

Most of the CRAN and Eclipse interviewees that we asked, felt that it was better to have fewer dependencies. Reasons for limiting dependencies included limiting one's exposure to upstream changes and not burdening one's users with a lot of modules to install and potential version conflicts ("dependency hell"). One Eclipse interviewee represents a common view: I only depend on things that are really worthwhile. Because basically everything that you depend on is going to give you pain every so often. And that's inevitable. Apart from removing no longer needed dependencies (tooling provided in Eclipse), some developers took more aggressive actions to avoid dependencies, including copying or recreating the functionality of another package.

In contrast, due to Node.js/npm's ability to use old versions and Eclipse's stability, some developers specifically said that they didn't see dependencies as a burden.

Selecting appropriate dependencies

When limiting themselves to appropriate dependencies, interviewees mentioned a variety of different signals they looked for, such as trust of the developers, activity levels in the project, size and identiy of the user base, the project's history, and the quality of project artifacts.

Encapsulating from change

Interestingly, there was almost no mention of traditional encapsulation strategies to isolate the impact of changes to upstream modules, contra to our expectations and typical software-engineering teaching. Only one Node.js developer mentioned developing an abstraction layer between his package and an upstream dependency.

Perhaps the coordination cost of following upstream packages, learning about changes, and understanding their impact is great enough that it dwarfs the code editing cost needed to make many changes in tightly coupled use of an upstream API.