FederationStats

= Federation usage statistics =

Federation visualization project
 * source (ruby on rails) https://repo.niif.hu/gitweb/gitweb.cgi?p=federation-stats.git;a=summary
 * live demo http://eduid.hu/stats

Running the sample project

 * Install Ruby
 * Install Rails
 * Setup a  database with the   command
 * Fire up, it will run the project on localhost:3000

Statistic types
Currently we have the following types of statistics:
 * Unique users per day
 * AuthnResponse per day
 * AuthnResponse per service per day

Log statistics format
The following simple format is used to convey statistics from IdPs to the central module - the white spaces (new lines) are important:

ENTITYID #ENTITYID# APIKEY #API_KEY# DATE yyyy-mm-dd STAT #STAT_ID# xxxx STAT #STAT_ID# yyyy STAT #STAT_ID# ww | #PEER_ENTITY_1# zz | #PEER_ENTITY_2#

The following sample might help understanding the format:

ENTITYID https://idp.niif.hu/idp/shibboleth APIKEY 0123....... DATE 2009-03-18 STAT AUTH 68 logins STAT USER_COUNT 16 unique userids STAT SSO_TO_SERVICE 1       | urn:geant:niif.hu:niifi:sp:register.ca.niif.hu 12       | https://repo.niif.hu/shibboleth 1       | https://sandbox.aai.niif.hu/shibboleth 5       | https://sysmonitor.hbone.hu/shibboleth 10      | https://www.ki.iif.hu/shibboleth 1       | https://noc6.vh.hbone.hu/shibboleth 21      | https://webadmin.iif.hu/shibboleth 3       | https://rrd-ma.perfsonar.vh.hbone.hu/shibboleth 7       | https://ugyeletes.vh.hbone.hu/shibboleth 2       | https://noc.grid.niif.hu/shibboleth 1       | https://wiki.voip.niif.hu/shibboleth 2       | https://netmonitor.hbone.hu/shibboleth 2       | https://idp.sch.bme.hu:443/opensso/sp/test

Running the log statistics collector
This following script can be used the collect statistics from the idp audit logs of Shibboleth 2 IdP generated on the day before running. It is based on Peter Schober's audit_r7.py, and good for run from daily cronjob:

The script below can be used the collect statistics from all the idp audit logs placed in a folder.

Feeding the database with the statistics
The federation statistics rails project contains the  command which can process the statistics format and load the data to the database. Note that this script currently contains an absolute path for the  script, so you must fix this before use.

Using HTTP-Post to feed the database
When deployed, the rails project provides a  URL to which one could POST the generated statistics file.

Creating IdPs
Use the rails console to create new idps:

$ RAILS_ENV=production script/console >> Entity.create :name => 'foo', :entity_type => 'idp' => #