Hortonworks has formed an unlikely alliance with rival Pivotal for developing the open-source Ambari project.

The alliance, which is meant to improve enterprise-grade solutions on Hortonworks' Apache Ambari software, is widely considered as a challenge to Cloudera, the Intel-backed big data company that is the leader in Hadoop-based software. 

Ambari is designed for managing and provisioning Apache Hadoop clusters. It has a dashboard that can be used to monitor whether a Hadoop cluster is operational. The software also enables users to reconfigure Hadoop clusters. As part of the agreement between the two companies, Pivotal will lend its engineers to improve Ambari's management and configuration capabilities.

"Today we are excited to announce that Pivotal and Hortonworks will collaborate on the Apache Ambari project to help strengthen Hadoop as an enterprise offering and to further advance the Apache Hadoop ecosystem. Apache Hadoop projects are central to our efforts to drive the most value for the enterprise. An open source, extensible and vendor-neutral application to manage services in a standardized way benefits the entire ecosystem," Jamie Buckley, Pivotal's vice president of product management for data fabric, said in a blog post.

The goal of the two companies is to alter Ambari so that the software can manage large Hadoop clusters more easily. In an interview, Hortonworks vice president of corporate strategy Shaun Connolly said that his company and Pivotal will combine their engineering efforts to the duplication of the work. Currently, Ambari is able to manage thousands of nodes. However, Connolly said that its capacity can be increased.

In spite of signing on for an alliance with Hortonworks, Pivotal will continue to work on and offer its Pivotal Command Center, the software that is included in the company's HD Hadoop distribution. In another interview, a spokesperson for Pivotal said the company will reveal its plans for the Pivotal Command Center and Ambari by the third quarter.

The alliance with Pivotal is definitely beneficial for Hortonworks, which has its own open-source Hadoop solution called Hortonworks Data Platform. The agreement is expected to boost its prospects, especially considering that HP just invested $50 million in the company.