Text File

Xref: feenix.metronet.com comp.infosystems.gopher:4460 Newsgroups: comp.infosystems.gopher Path: feenix.metronet.com!news.utdallas.edu!tamsun.tamu.edu!cs.utexas.edu!uwm.edu!cs.uwp.edu!datta From: datta@cs.uwp.edu (David Datta) Subject: new Log stats program Message-ID: <1993Aug14.064250.9293@cs.uwp.edu> Organization: University of Wisconsin - Parkside X-Newsreader: TIN [version 1.2 PL1] Date: Sat, 14 Aug 1993 06:42:50 GMT Lines: 350

I have just finished modifying the xferstats program that analyzes data from wuarchive's ftpd program to also handle Gopher logfiles.

It only does stats on Gopher file transfers (it doesn't do stats on directory requests.) It turns out, the music archives gets about 18,000 files sent out a day.

This xferstats is quite different from the one that is distributed, I take the blame for all bugs and I appreciate comments.

Edit the defaults at the top for your sites....

(Don't forget to clip the signature from the end...)

----CUT HERE ----------------------------------------------------------------- #! /usr/local/bin/perl # --------------------------------------------------------------------------- # # USAGE: xferstats <options> # # OPTIONS: # -f <filename> Use <filename> for the log file # -r include real users # -a include anonymous users # -h include report on hourly traffic # -d include report on domain traffic # -t report on total traffic by section # -D <domain> report only on traffic from <domain> # -l <depth> Depth of path detail for sections # -s <section> Section to report on, For example: -s /pub will report # only on paths under /pub # -g Process a Gopher file, not an FTP file # # ---------------------------------------------------------------------------

# edit the next two lines to customize for your domain. # This will allow your domain to be seperated in the domain listing.

= "uwp"; = "edu";

# edit the next line to customize for your default log file = "/usr/adm/xferlog"; = "/usr/adm/gopherlog";

# Edit the following lines for default report settings. # Entries defined here will be over-ridden by the command line.

= 1; = 0; = 1; = 3;

require 'getopts.pl'; &Getopts('f:rahdD:l:s:g');

if () { = 1;} if () { = 1;} if ( == 0 && == 0) { = 1; } if () { = ;} if () { = ;}

open (LOG,) || die "Error opening usage log file: \n";

if () {print "This report is for GOPHER transfers.\n";} else {print "This report is for FTP transfers.\n";}

if () {print "Transfer Totals include the '' domain only.\n"; print "All other domains are filtered out for this report.\n\n";}

if () {print "Transfer Totals include the '' section only.\n"; print "All other sections are filtered out for this report.\n\n";}

line: while (<LOG>) {

@line = split; if (!) { next if ($#line != 16); next if (! && [12] eq "a"); next if (! && [12] eq "r"); }

= substr(, 0, 10) . substr(, 19, 5); = substr(,11,2);

if () { if ( [9] eq "range") { =[12]-[10]; =[15]; } else { {next if ( [9] ne "file");} =[10]; = (-s "/usr/ftp/".[10]); }} else { =[8]; = [7]; if ( eq "\.") { = "/unreadable/filename";} }

next if (substr(,0,length("")) ne ""); = substr(,length("")); @path = split(/\//, );

# Things in the top-level directory are assumed to be informational files

if ($#path == 1) { = "Index/Informational Files"; } else { = ""; for (=1; <= $#path-1 && <= ;++) { = . "/" . []; } }

if () {=[7];} else {=[6];}

=~ tr/A-Z/a-z/;

@address = split(/\./, );

= [$#address]; if ( eq "" && [$#address-1] eq "") { = . "." . ; } if ( int([0]) > 0 || $#address < 2 ) { = "unresolved"; }

= 1; if () {if (substr(,0,length("")) eq "" ) { = 1;} else { = 0;} }

if () {

++; # total files sent ++; # total files sent {}++; # files per day {}++; # per-group accesses {}++;

{} += [5]; # xmit seconds per day {} += [5]; # xmit seconds for domain {} += ; # bytes per day {} += [7]; # xmit bytes to domain += ; # total bytes sent {} += ; # per-group bytes sent

{}++; # files per hour {} += [5]; # xmit seconds per hour {} += ; # bytes per hour += ; # total bytes sent } } close LOG;

@syslist = keys(systemfiles); @dates = sort datecompare keys(xferbytes);

if ( == 0) {die "There was no data to process.\n";}

print "TOTALS FOR SUMMARY PERIOD ", [0], " TO ", [$#dates], "\n\n"; printf ("Files Transmitted During Summary Period %12.0f\n", ); printf ("Bytes Transmitted During Summary Period %12.0f\n", ); printf ("Systems Using Archives %12.0f\n\n", $#syslist+1);

printf ("Average Files Transmitted Daily %12.0f\n", / ($#dates + 1)); printf ("Average Bytes Transmitted Daily %12.0f\n", / ($#dates + 1));

format top1 =

Daily Transmission Statistics

Number Of Number of Average Percent Of Percent Of Date Files Sent Bytes Sent Xmit Rate Files Sent Bytes Sent --------------- ---------- ----------- ---------- ---------- ---------- .

format line1 = @<<<<<<<<<<<<<< @>>>>>>>>> @>>>>>>>>>> @>>>>>>>>> @>>>>>>> @>>>>>>> , , , , , .

$^ = top1; $~ = line1;

foreach ( sort datecompare keys(xferbytes) ) {

= {}; = {}; = sprintf("%5.1f KB/s", {}/{}/1000); = sprintf("%8.2f", 100*{} / ); = sprintf("%8.2f", 100*{} / ); write; }

if () { format top2 =

Total Transfers from each Archive Section (By bytes)

---- Percent Of ---- Archive Section Files Sent Bytes Sent Files Sent Bytes Sent ------------------------- ---------- ----------- ---------- ---------- .

format line2 = @<<<<<<<<<<<<<<<<<<<<<<<< @>>>>>>>>> @>>>>>>>>>> @>>>>>>> @>>>>>>> , , , , .

$| = 1; $- = 0; $^ = top2; $~ = line2;

foreach ( sort bytecompare keys(groupfiles) ) {

= {}; = {}; = sprintf("%8.2f", 100 * {} / ); = sprintf("%8.2f", 100 * {} / ); write;

}

if ( < 1 ) { = 1; } if ( < 1 ) { = 1; } }

if () { format top3 =

Total Transfer Amount By Domain

Number Of Number of Average Percent Of Percent Of Domain Name Files Sent Bytes Sent Xmit Rate Files Sent Bytes Sent ----------- ---------- ------------ ---------- ---------- ---------- .

format line3 = @<<<<<<<<<< @>>>>>>>>> @>>>>>>>>>>> @>>>>>>>>> @>>>>>>> @>>>>>>> , , , , , .

$- = 0; $^ = top3; $~ = line3;

foreach ( sort domnamcompare keys(domainfiles) ) {

if ( {} < 1 ) { {} = 1; }

= {}; = {}; = sprintf("%5.1f KB/s", {}/{}/1000); = sprintf("%8.2f", 100 * {} / ); = sprintf("%8.2f", 100 * {} / ); write;

}

print "\n"; print "These figures only reflect ANONYMOUS FTP transfers. There are many\n"; print "sites which mount the archives via NFS, and those transfers are not\n"; print "logged and reported by this program.\n\n";

}

if () {

format top8 =

Hourly Transmission Statistics

Number Of Number of Average Percent Of Percent Of Time Files Sent Bytes Sent Xmit Rate Files Sent Bytes Sent --------------- ---------- ----------- ---------- ---------- ---------- .

format line8 = @<<<<<<<<<<<<<< @>>>>>>>>> @>>>>>>>>>> @>>>>>>>>> @>>>>>>> @>>>>>>> , , , , , .

$| = 1; $- = 0; $^ = top8; $~ = line8;

foreach ( sort keys(xfertbytes) ) {

= {}; = {}; = sprintf("%5.1f KB/s", {}/{}/1000); = sprintf("%8.2f", 100*{} / ); = sprintf("%8.2f", 100*{} / ); write; } } exit(0);

sub datecompare {

= substr(, 11, 4) * 4800; = substr(, 11, 4) * 4800; += index("JanFebMarAprMayJunJulAugSepOctNovDec",substr(, 4, 3))*400; += index("JanFebMarAprMayJunJulAugSepOctNovDec",substr(, 4, 3))*400; += substr(, 8, 2); += substr(, 8, 2); - ;

}

sub domnamcompare {

= length() - length(); ( < 0) ? -1 : ( > 0) ? 1 : ( lt ) ? -1 : ( gt ) ? 1 : 0;

}

sub bytecompare {

= {} - {}; ( < 0) ? -1 : ( > 0) ? 1 : ( lt ) ? -1 : ( gt ) ? 1 : 0;

}

sub faccompare {

= {} - {}; ( < 0) ? -1 : ( > 0) ? 1 : ( lt ) ? -1 : ( gt ) ? 1 : 0;

}

----CUT HERE ----------------------------------------------------------------- -- - Dave Datta@cs.uwp.edu

"Information wants to be free." - Elektric Music