AWstats
Toores. Ehk seda pala võib täiendada.
Sisukord
Sissejuhatus
AWStats on vabavaraline võimalusterohke abivahend veebi, ftp või postiserver graafilise statistika genereerimiseks.
Paigaldamine ja kasutamine FreeBSD keskkonnas
# /usr/ports/www/awstats && make install clean
misjärel awstats soovitab lisada /usr/local/etc/apache22/httpd.conf faili järgneva info
* Use /usr/local/www/awstats/tools/awstats_configure.pl to setup your hosts.
* Documentation has been installed in /usr/local/share/doc/awstats
*****************************************************************
Please add the following to your apache config, and restart.
#
# Directives to allow use of AWStats as a CGI
#
Alias /awstatsclasses "/usr/local/www/awstats/classes/"
Alias /awstatscss "/usr/local/www/awstats/css/"
Alias /awstatsicons "/usr/local/www/awstats/icons/"
ScriptAlias /awstats/ "/usr/local/www/awstats/cgi-bin/"
#
# This is to permit URL access to scripts/files in AWStats directory.
#
<Directory "/usr/local/www/awstats/">
Options None
AllowOverride None
Order allow,deny
Allow from all
</Directory>
Seejärel
# /usr/local/etc/rc.d/apache22 restart # cd /usr/local/www/awstats/ # mkdir /usr/local/etc/awstats # cp awstats.model.conf /usr/local/etc/awstats/
Lihtne konfiguratsioon tekitada kaustadesse kas /etc/awstats võo /usr/local/etc/awstats nimega awstas.zoo.tartu.ee.conf
LogFile="/srv/log/serverid/zoo/http_access.log" DirIcons="/awstatsicons" SiteDomain="zoo.tartu.ee" LogFormat=1 DNSLookup = 1 HostAliases = "www.zoo.tartu.ee"
Selle käivitamine
/usr/bin/perl /usr/local/www/awstats/cgi-bin/awstats.pl -config=zoo.tartu.ee -update
tulemus
# /usr/local/www/awstats/cgi-bin/awstats.pl -config=zoo.tartu.ee -update Create/Update database for config "/usr/local/etc/awstats/awstats.zoo.tartu.ee.conf" by AWStats version 7.0 (build 1.971) From data in log file "/srv/log/serverid/zoo/http_access.log"... Phase 1 : First bypass old records, searching new record... Searching new records from beginning of log file... Phase 2 : Now process new records (Flush history on disk after 20000 hosts)... Jumped lines in file: 0 Parsed lines in file: 139205 Found 0 dropped records, Found 0 comments, Found 0 blank records, Found 25862 corrupted records,log Found 0 old records, Found 113343 new qualified records.
Statistikat näeb veebist aadressilt:
Paljude domeenide korral võib näidata ka:
HostAliases="REGEX[.]"
Ajaloofailid iga kuu kohta tekitatakse fail kausta /usr/local/www/awstats/cgi-bin
awstats032011.havike.eenet.ee.txt awstats042011.havike.eenet.ee.txt awstats052011.havike.eenet.ee.txt awstats062011.havike.eenet.ee.txt awstats072011.havike.eenet.ee.txt awstats082011.havike.eenet.ee.txt awstats092011.havike.eenet.ee.txt awstats102011.havike.eenet.ee.txt awstats112011.havike.eenet.ee.txt
Igaüks neist sisaldab ühe kuu ajaloo mille alusel cgi veebiliides mätsib kokku kena pildi
Awstatsi paigaldamine Linux keskonnas
TODO
Erinevatest logiformaatidest ja nende kohendamisest
Here are the Awstats LogFormat values:
- - Apache or Lotus Notes/Domino native combined log format (NCSA combined/XLF
- - IIS or ISA format (IIS W3C log format). See FAQ-COM115 For ISA.
- - Webstar native log format.
- - Apache or Squid native common log format (NCSA common/CLF log format)
Soovitatav on seadistada konfiguratsioonis veel EnableLockForUpdate=1 et mitu awstatsi protsessi ei saaks korraga käivituda ning andmebaasi faile ära solkida.
Mida teha kui logiformaat erineb ülal kirjeldatutest
Lihtne kuid raske pähkel, syslog kirjutab kuskilt teisest masinast logiserverisse saadetava logi puhul näiteks täiendavat infot ning LogFormat ei sobi enam awstatsi jaoks. Kõige lihtsam on seljuhul muuta syslogi templaati, et kirjutataks koheselt kettale õiget logi. Kui aga seda pole varem tehtud ja vaja ka vanas formaadis logid läbi luristada, tuleb pisut häkkida.
Logiread näevad näiteks välja sellied
Apr 1 00:04:35 zoo1 logger: zoo.edu.ee 46.17.96.19 - - [01/Apr/2011:00:04:35 +0300] "GET / HTTP/1.0" 200 563 "http://zoo.edu.ee/" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; MyIE2; Deepnet Explorer)"
Juures on üleliigne esimene osa Apr 1 00:04:35 zoo1 logger: zoo.edu.ee
Tavaline common logi (logformat=1) võiks välja näha
111.22.33.44 - - [10/Jan/2001:02:14:14 +0200] "GET / HTTP/1.1" 200 1234 "http://www.fromserver.com/from.htm" "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)"
Apache native logi (logformat=4) aga järgnev
87.98.36.10 - - [28/Feb/2011:17:40:58 +0200] "GET /favicon.ico HTTP/1.1" 200 1406
Selle saab eemaldada customiseerides logi
LogFormat="%other %other %other %other %other %other %host %other %logname %time1 %methodurl %code %bytesd %refererquot %uaquot"
Siin tekib aga kohe uus probleem, nimelt kuupäevadel 1-9 on topelttühik. Seega teatab awstats Corrupted record. Ridu näeb täpsemalt lisades pramaeetri −showcorrupted. Seega tuleb eemaldada enne statistika loomist kõik topelttühikud.
$ sed 's/ */ /g' suuraprill.log >> suuraprill2.log
Veel üks näide:
LogFormat = "%virtualname %host %other %logname %time1 %methodurl %code %bytesd %refererquot %uaquot"
ja logirida
hezahur.havike.eenet.ee 69.4.231.200 - - [10/Nov/2011:04:20:45 +0200] "GET /koduleht/?q=comment/reply/86 HTTP/1.1" 200 28773 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
Nginx veebiserveri puhul võiks olla awstatsi conf selline
LogFormat="%host %other %logname %time1 %methodurl %code %bytesd %refererquot %uaquot"
Muutujate täielik ingliskeelne loend
- %host Client hostname or IP address (or Sender host for mail log)
- %host_r Receiver hostname or IP address (for mail log)
- %lognamequot Authenticated login/user with format: "alex"
- %logname Authenticated login/user with format: alex
- %time1 Date and time with format: [dd/mon/yyyy:hh:mm:ss +0000] or [dd/mon/yyyy:hh:mm:ss]
- %time2 Date and time with format: yyyy-mm-dd hh-mm-ss
- %time3 Date and time with format: Mon dd hh:mm:ss or Mon dd hh:mm:ss yyyy
- %time4 Date and time with unix timestamp format: dddddddddd
- %methodurl Method and URL with format: "GET /index.html HTTP/x.x"
- %methodurlnoprot Method and URL with format: "GET /index.html"
- %method Method with format: GET
- %url URL only with format: /index.html
- %query Query string (used by URLWithQuery option)
- %code Return code status (with format for web log: 999)
- %bytesd Size of document in bytes
- %refererquot Referer page with format: "http://from.com/from.htm"
- %referer Referer page with format: http://from.com/from.htm
- %uaquot User agent with format: "Mozilla/4.0 (compatible, ...)"
- %ua User agent with format: Mozilla/4.0_(compatible...)
- %gzipin mod_gzip compression input bytes: In:XXX
- %gzipout mod_gzip compression output bytes & ratio: Out:YYY:ZZpct.
- %gzipratio mod_gzip compression ratio: ZZpct.
- %deflateratio mod_deflate compression ratio with format: (ZZ)
- %email EMail sender (for mail log)
- %email_r EMail receiver (for mail log)
- %virtualname Web sever virtual hostname. Use this tag when same log
- contains data of several virtual web servers. AWStats
- will discard records not in SiteDomain nor HostAliases
- %cluster If log file is provided from several computers (merged by
- logresolvemerge.pl), this tag define field of cluster id.
- %extraX Another field that you plan to use for building a
- personalized report with ExtraSection feature (See later).
- If your log format has some fields not included in this list, use
- %other Means another field
- %otherquot Means another not used double quoted field