During the last InnerSource Commons in Baltimore, Daniel Izquierdo Cortázar and I had the opportunity to introduce the CHAOSS Project to InnerSource practitioners. This post summarizes the main topics we covered.
What is CHAOSS, anyway?
CHAOSS is a project from the Linux Foundation that started in 2017, founded by well-known companies such as Red Hat, Mozilla, Eclipse Foundation, and Bitergia. The name CHAOSS is short for Community Health Analytics for Open Source Software. The CHAOSS Project focuses on creating analytics and metrics to help define community health.
InnerSource is about applying open source practices within an organization and the metrics and tools developed by CHAOSS for open source apply to InnerSource just the same. In fact, the metrics and tooling created in the CHAOSS project can be applied to any software development project, including open source and InnerSource projects. Therefore, it is interesting to understand better what CHAOSS is doing and how to leverage the best practices developed there.
The objectives of CHAOSS are:
- Establish a standard of metrics around community health.
- Create open source software for producing those community health metrics.
- Create a way to build consistent reporting around community health metrics.
“All this is because we need data to know what’s going on in our communities and what is going on within our projects because if we don’t have data, we don’t know anything for certain.”
CHAOSS structure: Working groups
At first, CHAOSS had two committees: a metrics committee and a software committee. The metrics committee was focused to establish standard implementation-agnostic metrics for measuring community activity, contributions, and health. The software committee was focused on producing integrated open source software in order to analyze software development projects.
As work on metrics and tooling progressed, the CHAOSS community formed working groups to focus on specific aspects of community health. Members from metrics and software committees work across all CHAOSS working groups: Diversity and Inclusion, Risk, Evolution, Value, and Common Metrics. The Common Metrics working group is trying to be a centralized place for common metrics from each of the groups, such as responsiveness to issues and pull requests or organizational engagement metrics.
CHAOSS Releases
In August 2019, CHAOSS for the first time released from each working group a set of metrics including specific definitions of the metrics and methods for collecting the metrics. As Ana explained during the talk:
“The way CHAOSS and Bitergia create metrics is not just like, ‘Okay, let’s go and add metrics like there is no tomorrow!’ Rather, CHAOSS follows the Goal-Question-Metric approach. First, CHAOSS defines goals and from that, starts asking questions related to those goals. Metrics are pointed towards these questions to provide the answers we seek. This process is repeated across the different working groups.”
The screenshot shows the main structure and topics the CHAOSS community is working on. Daniel also mentioned some similarities between CHAOSS’s way of working and InnerSource commons:
“If you think about how we are defining metrics at the InnerSource Commons, it sounds similar. We are trying to formalize metrics in this case finding InnerSource patterns. We are also trying to formalize patterns by having templates and trying to have all of these discussions within the community.”
Implementation: CHAOSS Software
Regarding the software used, CHAOSS has three main projects: Augur, Cregit, and GrimoireLab.
“GrimoireLab is the tool we know well because Bitergia is the main maintainer of the project. So we’ll take a look at GrimoireLab a bit more. If you’re interested in any of the other two, let me know.”
If you want to know more about GrimoireLab, you can always visit the GrimoireLab website, which details every software component involved and the supported data sources. As a reminder, this is a 100% open source project, so feel free to contribute and provide feedback.
“On the left side (Fig. 1), there are some examples of different data sources and then a set of tools for processing the data. For example, we have Perceval to collect data. GELK is the enrichment platform. Sorting Hat is the identity center affiliations platform. Kibiter, a downstream version of Kibana, provides visualizations and data exploration. And then at the end you can produce dashboards and reports.”
Is CHAOSS Software suitable for InnerSource?
CHAOSS is focused in open source but when we think about the metrics to measure Innersource initiatives, they are not much different.
There are certain things that we are interested in from an InnerSource perspective that can also be found in open source environments: We are talking about communication and collaboration in both cases.
We can reuse the same metrics and ways to measure this in an effort to improve communication and collaboration in InnerSource. The scope and focus is different between Open Source and InnerSource, but the tools and metrics can relate.
During the talk we introduced some InnerSource dashboards that Bitergia has been working on, addressing specific business goals and questions. They focused on three main areas: growth, engagement, and efficiency.
Daniel and I were very excited to share the dashboards and metrics at the InnerSource Commons meeting. However, this blog post is already long enough and we will publish this information in our next Bitergia blog post: “KPIs to measure InnerSource and internal collaboration.” To get a sneak peek, take a look at the Bitergia presentation slides.
Ideas for next step to improve metrics at ISC
Consistent documentation
What do you mean by active contributor? what do you mean by commit? what about collaboration or communication? InnerSource Commons needs a consistent definition of these terms to have a clear view and comparable insights.
There is already a metrics platform channel running, where InnerSource practitioners can share the metrics they’re applying within their Innersource initiative. Maybe next step can be building a place where companies that apply InnerSource could share their top metrics anonymously to allow benchmarking across industries.
Framing the metrics discussion
Indeed, metrics is a hot topic at ISC but there’s been no discussion beyond initial steps. Next step can be working on specific use cases and keep on discussing about InnerSource maturity models. Use cases contextualize the use of metrics and can help with sharing experiences and best practices. Maturity models are a way to formalize experiences with metrics and make them portable.
Building bridges between CHAOSS and ISC
CHAOSSCon Europe 2020 conference is around the corner, and this place can be a good opportunity to introduce InnerSource community to CHAOSS members, and learn from each other. The Call for Participation is open until November 30 and submissions from InnerSource metrics would be very welcomed there.
Bitergia will be attending to Sustain Summit, FOSDEM and CHAOSScon at the end of January and beginning of February. If you don’t want to miss a thing about InnerSource metrics evolution, subscribe to our blog for upcoming news!
