„FederationStats” változatai közötti eltérés

Innen: KIFÜ Wiki
(HTTP-Post feeding, api keys)
a (Running the log statistics collector)
63. sor: 63. sor:
  
 
== Running the log statistics collector ==
 
== Running the log statistics collector ==
This following script can be used the collect statistics from the idp audit logs of Shibboleth 2 IdP. It is based on Peter Scober's audit_r7.py:
+
This following script can be used the collect statistics from the idp audit logs of Shibboleth 2 IdP. It is based on Peter Schober's audit_r7.py:
  
 
<source lang="bash">
 
<source lang="bash">

A lap 2010. május 18., 15:32-kori változata

Federation usage statistics


Federation visualization project

Running the sample project

  • Install Ruby
  • Install Rails (gem install rails)
  • Setup a development.sqlite3 database with the rake db:setup command
  • Fire up script/server, it will run the project on localhost:3000

Statistic types

Currently we have the following types of statistics:

  • Unique users per day (USER_COUNT)
  • AuthnResponse per day (AUTH)
  • AuthnResponse per service per day (SSO_TO_SERVICE)

Log statistics format

The following simple format is used to convey statistics from IdPs to the central module - the white spaces (new lines) are important:

ENTITYID #ENTITYID#
APIKEY #API_KEY#
DATE yyyy-mm-dd

STAT #STAT_ID#
xxxx

STAT #STAT_ID#
yyyy

STAT #STAT_ID#
ww | #PEER_ENTITY_1#
zz | #PEER_ENTITY_2#

The following sample might help understanding the format:

ENTITYID https://idp.niif.hu/idp/shibboleth
APIKEY 0123.......
DATE 2009-03-18

STAT AUTH
68 logins

STAT USER_COUNT
16 unique userids

STAT SSO_TO_SERVICE
1        | urn:geant:niif.hu:niifi:sp:register.ca.niif.hu
12       | https://repo.niif.hu/shibboleth
1        | https://sandbox.aai.niif.hu/shibboleth
5        | https://sysmonitor.hbone.hu/shibboleth
10       | https://www.ki.iif.hu/shibboleth
1        | https://noc6.vh.hbone.hu/shibboleth
21       | https://webadmin.iif.hu/shibboleth
3        | https://rrd-ma.perfsonar.vh.hbone.hu/shibboleth
7        | https://ugyeletes.vh.hbone.hu/shibboleth
2        | https://noc.grid.niif.hu/shibboleth
1        | https://wiki.voip.niif.hu/shibboleth
2        | https://netmonitor.hbone.hu/shibboleth
2        | https://idp.sch.bme.hu:443/opensso/sp/test

Running the log statistics collector

This following script can be used the collect statistics from the idp audit logs of Shibboleth 2 IdP. It is based on Peter Schober's audit_r7.py:

#!/bin/bash

FILES="log-directory/idp-audit-*.log"
TARGETDIR="/tmp/stats"
ENTITYID="idp-entity-id"
APIKEY="aaa....."
AUDIT="/path/to/audit_r7.py"

for f in $FILES
do
        echo "Processing $f file..."
        DATE=${f:10:10}
        LOGINS=`$AUDIT -l $f`
        USERS=`$AUDIT -u $f`
        SERVICES=`$AUDIT -p $f | sed '/^[0-9]/p' -n`
        FILENAME="stat-$f"

        echo "ENTITYID $ENTITYID
APIKEY $APIKEY
DATE $DATE

STAT AUTH
$LOGINS

STAT USER_COUNT
$USERS

STAT SSO_TO_SERVICE
$SERVICES
" > $TARGETDIR/$FILENAME

done #for files

Feeding the database with the statistics

The federation statistics rails project contains the script/stat_parser/file.rb command which can process the statistics format and load the data to the database. Note that this script currently contains an absolute path for the script/runner script, so you must fix this before use.

Using HTTP-Post to feed the database

When deployed, the rails project provides a /import_stats URL to which one could POST the generated statistics file.

Creating IdPs

You must create records for the IdPs from which the application accepts statistics. This could be done via the /entities/new URL. The name of the entity must be the entityID, the type must be 'idp'.

API keys

Every IdP record in the database will contain a 40 character length API key.