Londiste

Sissejuhatus

Londiste http://wiki.postgresql.org/wiki/Londiste_Tutorial on asünkroonne master/slave-tüüpi andmete replitseerimislahendus. Londiste tarkvara sisaldub Skytools komplektis ühe rakendusena.

Tööpõhimõte

Londiste kasutab PGQ http://wiki.postgresql.org/wiki/PGQ_Tutorial lahendust andmete asünkroonseks replikeerimiseks. Londiste on

• nn 'master to many slaves' lahendus
• sünkroniseeritakse tabeleid (sh loendureid)

Londiste töötamise võiks kujutada sellise skeemiga

                    MASTER                                   SLAVE
  
                  -----|---------------------------------------|-----
                      _|_                                     _|_
                     |   |                                   |   |
                     |___|                                   |___|
 
 seadistusfail:    ticker.ini                              p-to-s.ini
      protsess:    pgqadm ticker.ini ticker                londiste p-to-s.ini replay
       skeemid:    pgq, londiste                           londiste

kus

• MASTER - andmebaas, mille tabelites kasutajad muudavad andmeid
• SLAVE - andmebaas, mille tabelite sisu hoitakse masteriga sünkroonis
• pgq, londiste - replikeerimisega seotud skeemid; selliste nimedega skeeme ei tohiks andmebaasis muuks kasutuseks olla

Londiste paigaldamine

Londiste paigaldamiseks Debian Squeeze keskkonnas sobib öelda

# apt-get install skytools

Paigaldamise tulemusena lisanduvad muu hulgas failid

/usr/bin/walmgr
/usr/bin/table_dispatcher
/usr/bin/skytools_upgrade
/usr/bin/scriptmgr
/usr/bin/queue_splitter
/usr/bin/queue_mover
/usr/bin/pgqadm
/usr/bin/londiste
/usr/bin/cube_dispatcher
/usr/bin/bulk_loader

Londiste seadistamine ja kävitamine

Londiste tuleb seadistada mõlemas arvutis

• master andmebaasis tuleb käivitada pqg protsess
• slave andmebaasis tuleb käivitada londiste protsess

Master

$ cat /etc/londiste/ticker.ini 
[pgqadm]

# should be globally unique
job_name = pgqadm_londiste

db = dbname=londiste port=5432 host=127.0.0.1 user=postgres password=trust

# how often to run maintenance [minutes]
maint_delay_min = 5

# how often to check for activity [secs]
loop_delay = 0.1

logfile = ~/log/%(job_name)s.log
pidfile = ~/pid/%(job_name)s.pid

use_skylog = 0

Slave

$ cat /etc/londiste/londiste.ini 
[londiste]
job_name = test_to_subcriber

# source queue location
provider_db = dbname=londiste port=5432 host=192.168.10.205 user=postgres password=trust

# target database - it's preferable to run "londiste replay"
# on same machine and use unix-socket or localhost to connect
subscriber_db = dbname=londiste port=5432 host=127.0.0.1

# source queue name
pgq_queue_name = londiste.replica

logfile = ~/log/%(job_name)s.log
pidfile = ~/pid/%(job_name)s.pid

# how often to poll event from provider
#loop_delay = 1

# max locking time on provider (in seconds, float)
#lock_timeout = 10.0

Londiste kasutamine

TODO

• ticker.ini
• p-to-s.ini

ticker.ini

Londiste kasutamiseks tuleb tekitada seadistusfail ticker.ini

 $ cat /home/postgres/londiste/ticker.ini
 [pgqadm]
 job_name = myticker_name
 db = dbname=baasinimi host=10.0.13.170 user=postgres password=parool port=5432
 # how often to run maintenance [seconds]
 maint_delay = 600
 
 # how often to check for activity [seconds]
 loop_delay = 0.1
 logfile = ~/londiste/log/%(job_name)s.log
 pidfile = ~/londiste/pid/%(job_name)s.pid

kusjuures ühendussõnet saab testida selliselt

 $ python
 ...
 >>> import psycopg2
 >>> con = psycopg2.connect('dbname=baasinimi host=10.0.13.170 \
   user=postgres password=parool port=5432')
 >>> Ctrl-D

PGQ baasi osa paigaldamiseks tuleb seejärel öelda

 $ mkdir /home/postgres/londiste/log
 $ mkdir /home/postgres/londiste/pid
 $ export PYTHONPATH=/usr/local/skytools/lib/python2.4/site-packages 
 $ /usr/local/skytools/bin/pgqadm.py /home/postgres/londiste/ticker.ini install
 2009-11-03 13:28:41,793 28491 INFO plpgsql is installed
 2009-11-03 13:28:41,795 28491 INFO txid_current_snapshot is installed
 2009-11-03 13:28:41,796 28491 INFO Installing pgq
 2009-11-03 13:28:41,796 28491 INFO Reading from \
   /usr/local/skytools/share/skytools/pgq.sql

Tulemusena tekib andmebaasi baasinimi skeem pgq.

Tickeri käivitamiseks tuleb öelda

 $ /usr/local/skytools/bin/pgqadm.py /home/postgres/londiste/ticker.ini ticker -d

ja jääb käima vastav protsess.

p-to-s.ini seadistamine

 $ cat p-to-s.ini 
 [londiste]
 job_name = test_to_subcriber
 
 provider_db = dbname=portaal host=10.0.13.170 user=postgres \
   password=parool port=5433
 subscriber_db = dbname=portaal host=10.0.13.177 user=postgres \
   password=parool port=5432
 
 # it will be used as sql ident so no dots/spaces
 pgq_queue_name = londiste.replica
 
 logfile = /home/postgres/londiste/log/%(job_name)s.log
 pidfile = /home/postgres/londiste/pid/%(job_name)s.pid

Logiserverist öelda

 $ londiste.py p-to-s.ini provider install
 $ londiste.py p-to-s.ini subscriber install

ning käivitada deemon

 $ londiste.py p-to-s.ini replay -d

Andmete replitseerimiseks tuleb logiserverisse moodustada vastavad tableid ning seejärel öelda logiserverist

 $ londiste.py p-to-s.ini provider add public.T1 public.T2
 $ londiste.py p-to-s.ini subscriber add public.T1 public.T2

Subscriberi eemaldamiseks sobib öelda

 $ londiste.py p-to-s.ini subscriber remove skeeminimi.tabelinimi

Staatust saab küsida

 $ londiste.py p-to-s.ini subscriber tables
 Table                         State
 skeeminimi.tabelinimi         in-copy