What is Apache Twill?

Apache Twill is an abstraction over Apache Hadoop® YARN that reduces the complexity of developing distributed applications, allowing developers to focus more on their application logic. Apache Twill allows you to use YARN’s distributed capabilities with a programming model that is similar to running threads.

Why do I need Apache Twill?

Apache Twill dramatically simplifies and reduces development efforts, enabling you to quickly and easily develop and manage distributed applications through its simple abstraction layer on top of YARN. YARN, although originally designed for MapReduce v2, can be used as a generic cluster resource management framework that can run almost any type of applications on a Hadoop® cluster. Given it’s power, however, YARN can be quite difficult to use and requires a big ramp up effort since YARN’s capabilities are quite low level and the learning curve is really steep. Moreover, many distributed applications have common needs such as application lifecycle management, service discovery, distributed process coordination and resiliency to failure, which are not out of the box features from YARN.

Apache Twill allows you to develop, deploy and manage your distributed applications with a much simpler programming model, with rich build-in features for common distributed applications needs. Whether you are a developer or operating engineer will find Apache Twill helps you greatly reduces the effort in developing and operating your applications on a Hadoop® cluster.

Latest Release

The latest release of Apache Twill is 0.6.0-incubating. It comes with a lot of new features and improvements over the previous release. Please go to the release page to find out more.

Is it Building?

Status of CI build at Travis CI: Build Status

Apache Twill Presentation in ApacheCon 2014


Apache Twill is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.