Open Source Community Metrics for FOSDEM

Open Source Community Metrics
Tips and Techniques for Measuring Participation

FOSDEM 2013

Dawn M. Foster
Community Lead at Puppet Labs

@geekygirldawn
dawn@puppetlabs.com
puppetlabs.com

Metrics are Useful for Open Source Projects

● Measure progress in your community over time
● Who contributes
● Where are people contributing
● Spot trends
● Gauge interest
● Learn more about key contributors
● Recognize contributions

2

Example: Components of an Open Source Community

3 http://www.flickr.com/photos/korou/2586472234

Example: Puppet Metrics December 2012
Summary
5131 members and 887 messages in Puppet-Users
941 members and 108 messages in Puppet-Dev
919 nicks on #puppet IRC channel
1942 Puppet Forge accounts and 726 modules
3728 Redmine accounts
444 forks / 1082 watchers of Puppet

Six Month Comparison (July 2012)
4420 members and 1198 messages in Puppet-Users
830 members and 120 messages in Puppet-Dev
873 nicks on #puppet IRC channel
1405 Puppet Forge accounts and 442 modules
3064 Redmine accounts
342 Forks / 904 watchers of Puppet

http://PuppetLabs.com/community/metrics
4

Example: June 2011 MeeGo Community Metrics Summary
• 439,963 unique people visited this month (430,070 last month).
– 1,796,288 pageviews (1,857,907 last month); 599,573 total visits (597,658 last month)
• 25,801 people are members of MeeGo.com (was 24,019 last month)
– Dev ML subscribers = 4772; Community = 3736; iL10N = 2607; SDK = 3197
• Mailing Lists: 5890 posts this month; 200 people posted 2+ msgs
• Forums: 2255 posts. 242 people posted 2+ messages
• New Bugs Created: 1938; Bugs Resolved: 2871
• Downloads: 52,627 1.2 Netbook, 626 1.2 IVI, 1505 1.1 N900, 3072 Tablet
• Active Users: Estimated at 800 – 1000 people.
– Mailing Lists: 302 people with unique email addresses posted (308 last month)
– Forums: 412 people posted at least one item (262 last month)
– Bugzilla: 773 people performed some action (624 last month)
– Wiki: 276 edited the wiki (205 last month)
– IRC: 410-520 people logged into #meego simultaneously most days

http://wiki.meego.com/Metrics
5

What are the Right Metrics for YOUR Project

● Goals
● What are your overall goals for the project?
● How can you measure progress toward those goals?
● What is important to you and your progress?
● Trends
● What should you measure to recognize trends?
● How do you recognize when something is going wrong?
● Do you notice big improvements?

Note: I measure way too much
6 http://www.flickr.com/photos/bandfan/5548675317/

Mailing Lists: mlstats

Mailing List Stats is a command line tool used to analyze
mailing list archives. It downloads the archives, places
them in a directory and stores all the information contained
in each mailing list post into a database

http://libresoft.es/tools/mlstats

7

Mailing Lists: mlstats for LibreOffice Developer List
● Grab data from your mailing & store in db (repeat per ML)
– mlstats --db-user=root --db-password=
http://lists.freedesktop.org/archives/libreoffice/
● Top Content Query: All or By Month
– select subject, count(*) as c from messages group by subject
order by c;
– select subject,monthname(first_date) as m, year(first_date) as
y, count(*) as c from messages group by subject,
month(first_date) order by y, m, c;
● Top Poster Query
– select p.email_address,count(*) as c from messages as
m,messages_people as p where
m.message_id=p.message_ID group by p.email_address
order by c;

8

Mailing Lists: LibreOffice Top Content Result (graphed)
[Libreoffice] LibreOffice WikiHelp
[Libreoffice] Assertions and Logging
[ANN] Please use Gerrit from now on for Patch Review
[Bug 35673] LibreOffice 3.4 most annoying bugs
[Libreoffice] (no subject)
Static src analysis of LibreOffice
[Libreoffice] minutes of tech. steering call ...
[Libreoffice] minutes of tech steering call ...
License statement
No subject
[Libreoffice] [Bug 31865] [Task]: LibreOffice 3.3 release blockersnt/ stoppers
minutes of ESC call ...
Bugzilla Auto-posts
[Libreoffice] [Bug 37361] LibreOffice 3.5 most annoying bugs
[Libreoffice] [Bug 35673] LibreOffice 3.4 most annoying bugs

0 100 200 300 400 500 600 700

What are people talking about?
9

Mailing Lists: LibreOffice Top Posters Result
caolanm@redhat.com 2222
bugzilla-daemon@freedesktop.org 1929
michael.meeks@suse.com 1521
michael.meeks@novell.com 1434
sbergman@redhat.com 1224
gerrit@gerrit.libreoffice.org 1076
pmladek@suse.cz 972
nthiebaud@gmail.com 942
thb@documentfoundation.org 880
kendy@suse.cz 776
bjoern.michaelsen@canonical.com 723
mstahl@redhat.com 689
serval2412@yahoo.fr 638
kyoshida@novell.com 612
tml@iki.fi 556
dtardon@redhat.com 491
l.lunak@suse.cz 490
oolst@nouenoff.nl 474
timar74@gmail.com 473

Who contributes to ML discussions?
10

IRC: Multiple Analysis Tools Depending on Log Format

Generates IRC stats for active people, by hour of the day,
by day, most used words, quotes and more.

http://royale.zerezo.com/irssistats/
http://pisg.sourceforge.net/
http://code.google.com/p/superseriousstats/

11

Code Contributions: Ohloh

http://www.ohloh.net/p/puppet

14

New Contributors: Ohloh

http://www.ohloh.net/p/puppet

15

Recent Commits with Filter: Ohloh

http://www.ohloh.net/puppet

16

Bugs

● New bugs vs. resolved bugs
● Can't just look at monthly trends
● Need to take release cycle into account
● Before release: more resolved bugs
● After release: more new bugs
● Participants
● People who file new bugs
● Participate in bugs (comment, etc.)
● Careful with people who resolve bugs (usually QA)

Image: http://www.thegeekstuff.com/2010/05/install-bugzilla-on-linux/

17

Media Wiki

● Get Statistics
● http://wiki.meego.com/Special:Statistics
● http://wiki.meego.com/api.php?
action=query&meta=siteinfo&siprop=statistics&format=ya
mlfm

19

Automate: Scripts & Business Intelligence
● A less than elegant method (figure out what you want)
● Giant bash script
● Uses wget, awk, mysql queries, etc.
● Dumps a bunch of csv files on my hard drive
● A better dashboard approach (WIP)
● Open source metrics dashboard (business intelligence)
● Uses Pentaho for reporting, runs regularly and produces a
dashboard anyone can view at any time
● http://www.webdetails.pt/

20

Automate: Custom Code + Graphite
● What we have now at Puppet
● Lots of custom Ruby code
● 2 Projects:
● Gather: Gathers data from
various sources
● Metrics: Calls Gather and
displays output using Graphite
● Generates one image file per graph plus a yaml file with
min / max metadata
● Uses Graphite Render URL API to display data
● http://graphite.readthedocs.org/en/0.9.10/render_api.html
● I manually add arrows to explain bumps in graphs

21

Example: Custom Code + Graphite

22

Now What?

● Report Regularly
● Monthly – may be too often
● Quarterly? Yearly?
● Share the reports with the
community
● http://puppetlabs.com/community/metrics
● http://wiki.meego.com/Metrics
● https://www.tizen.org/community/community
-metrics
● Recognize
● Recognize your top contributors
http://www.flickr.com/photos/play4smee/2439494411/

23

Dawn Foster
Community Manager at Puppet Labs

@geekygirldawn

dawn@puppetlabs.com

http://www.flickr.com/photos/tlk/5630885373/

Open Source Community Metrics for FOSDEM

More Related Content

Similar to Open Source Community Metrics for FOSDEM (20)

More from Dawn Foster (20)

Recently uploaded (20)

Open Source Community Metrics for FOSDEM