Mar 042016
 

Installing on Centos 6

I need PostgreSQL 9.3 on my Centos 6 box

yum localinstall https://download.postgresql.org/pub/repos/yum/9.3/redhat/rhel-6-x86_64/pgdg-centos93-9.3-2.noarch.rpm

yum list postgres*
yum install postgresql93-server
service postgresql-9.3 initdb

chkconfig postgresql-9.3 
service postgresql-9.3 start
service postgresql-9.3 status

 

Login to database

sudo -i -u postgres
psql --username=postgres -l 
# list all databases on psql prompt
\list or \l
# list all tables in the current database
\dt

 

Enabling password auth

sudo vi /var/lib/pgsql/data/pg_hba.conf

Find the lines that looks like this, near the bottom of the file:

# pg_hba.conf excerpt (original)
host    all             all             127.0.0.1/32            ident
host    all             all             ::1/128                 ident

Then replace “ident” with “md5”, so they look like this:

# pg_hba.conf excerpt (updated)
host    all             all             127.0.0.1/32            md5
host    all             all             ::1/128                 md5

Save and exit. PostgreSQL is now configured to allow password authentication.

Extensions

IF you need extensions, like RegExp-optimized index extension pg_trgm, you are going to need postgresql-contrib package. Once you have it installed, run:

$ sudo -u postgres sh
$ psql puppetdb -c 'create extension pg_trgm'
$ exit

 

File based backup

service postgresql stop
tar -czf /backup/full_postgres_backup-`date +%Y%m%d`.tar.gz' /var/lib/pgsql/data
service postgresql start

Single DB dump, for example spacewalk database

su - postgres -c 'pg_dump rhnschema |gzip -c > /backup/rhnschema_postgres_backup-`date +%Y%m%d`.sql.gz'

Backing up from cron

Moar scripts here https://github.com/zmielna/backup_scripts

# crontab -l
01 3 * * * /root/bin/postgresql_dump.sh 2>&1 |logger

vim /root/bin/postgresql_dump.sh

#!/bin/bash
#############################################################################
# Simple script to create a snapshot of a PostgreSQL databases.
# Can be run from cron like that
# 01 3 * * * /root/bin/postgresql_dump.sh 2>&1 |logger
# 
# Send bugreports, fixes, enhancements, t-shirts, money, beer & pizza to [email protected]
#############################################################################

#------------ variables
# Directory to store backups in
DST=/backup/dbback_postgresql
# DATABASES="postgres rhnschema"
DATABASES=`su - postgres -c "psql --username=postgres -l -x"|grep Name|grep -v template|cut -d"|" -f2|xargs`

# Any backups older than this will be deleted first
KEEPDAYS=7
DATE=$(date  +%Y-%m-%d)
#------------ code
/bin/logger "Starting PostgreSQL Dump....."
# cd $DST
find ${DST} -type f -mtime +${KEEPDAYS} -exec rm -f {} \;
rmdir $DST/* 2>/dev/null
mkdir -p ${DST}/${DATE}
chown postgres. ${DST}/${DATE}
for db in $DATABASES ; do
        echo -n "Backing up ${db}... " | logger
	su - postgres -c  "pg_dump ${db} |gzip -c > ${DST}/${DATE}/${db}-`date +%Y%m%d`.sql.gz"
        echo -n "Done with $db." | logger
done
/bin/logger "OK, all PostgreSQL dumps done in $DST"
Feb 162016
 

Intro

Pistyll Rhaeadr

I needed a job scheduling system for a single machine, to allow group of people run some number crunching scripts. Decided to try SLURM and was surprised that there are no rpm repo/packages available for Centos – sadly that ain’t as easy as apt-get install slurm-llnl

But I managed to get it working in the end and here you can find a journal from this journey.

 

We need EPEL repo

rpm -Uvh http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm

Installing required bits and bobs

yum install -y munge-devel munge-libs readline-devel perl-ExtUtils-MakeMaker openssl-devel pam-devel rpm-build perl-DBI perl-Switch munge mariadb-devel

Downloading the latest stable version of Slurm

From http://www.schedmd.com/#repos

Building rpm packages

rpmbuild -ta slurm-15.08.7.tar.bz2

Once done install rpms

ls -l ~/rpmbuild/RPMS/x86_64/*.rpm
rpm -Uvh ~/rpmbuild/RPMS/x86_64/*.rpm

or even better upload it to your custom Spacewalk software channel. Don’t you use Spacewalk server? Check it out, if you have more Centos boxes then you gonna love it, it’s awesome.

We may also add user for slurm at that stage, we are going to need it at later.

useradd slurm
mkdir /var/log/slurm
chown slurm. /var/log/slurm

Install MariaDB

yum install mariadb-server -y
systemctl start mariadb
systemctl enable mariadb
mysql_secure_installation

# you can save mysql root password in root home dir,
# bad practise but from the other hand
# if someone can access root home dir
# then we are in troubles anyway

vim ~/.my.cnf
[client]
password = aksjdlowjedjw34dwnknxpw93e9032edwxbsx
# now root will have mysql root password-less shell.

Create SQL database

Start mysql shell and

mysql> grant all on slurm_acct_db.* TO 'slurm'@'localhost'
-> identified by 'some_pass' with grant option;
mysql> create database slurm_acct_db;

Configure SLURM db backend

# egrep -v '^#|^$' /etc/slurm/slurmdbd.conf
AuthType=auth/munge
DbdAddr=localhost
DbdHost=localhost
SlurmUser=slurm
DebugLevel=4
LogFile=/var/log/slurm/slurmdbd.log
PidFile=/var/run/slurmdbd.pid
StorageType=accounting_storage/mysql
StorageHost=localhost
StoragePass=some_pass
StorageUser=slurm
StorageLoc=slurm_acct_db

and enable service

systemctl start slurmdbd
systemctl enable slurmdbd
systemctl status slurmdbd

After starting service your shiny new database should be populated with tables:

MariaDB [slurm_acct_db]> show tables;
+-------------------------+
| Tables_in_slurm_acct_db |
+-------------------------+
| acct_coord_table |
| acct_table |
| clus_res_table |
| cluster_table |
| qos_table |
| res_table |
| table_defs_table |
| tres_table |
| txn_table |
| user_table |
+-------------------------+
10 rows in set (0.01 sec)

 

Time to configure Munge auth daemon

create-munge-key
systemctl start munge
systemctl status munge
systemctl enable munge

And finally the actual SLURM daemon

Stick something alongside these lines to your /etc/slurm/slurm.conf

# egrep -v '^#|^$' /etc/slurm/slurm.conf
ClusterName=efg
ControlMachine=efg01
SlurmUser=slurm
SlurmctldPort=6817
SlurmdPort=6818
AuthType=auth/munge
StateSaveLocation=/home/slurm/tmp
SlurmdSpoolDir=/tmp/slurmd
SwitchType=switch/none
MpiDefault=none
SlurmctldPidFile=/var/run/slurmctld.pid
SlurmdPidFile=/var/run/slurmd.pid
Proctracktype=proctrack/linuxproc
CacheGroups=0
ReturnToService=0
SlurmctldTimeout=300
SlurmdTimeout=300
InactiveLimit=0
MinJobAge=300
KillWait=30
Waittime=0
SchedulerType=sched/backfill
SelectType=select/linear
FastSchedule=1
SlurmctldDebug=3
SlurmdDebug=3
JobCompType=jobcomp/none
JobAcctGatherType=jobacct_gather/linux
JobAcctGatherFrequency=30
AccountingStorageType=accounting_storage/slurmdbd
NodeName=efg01 CPUs=16 State=UNKNOWN
PartitionName=debug Nodes=efg01 Default=YES MaxTime=INFINITE State=UP

and see if your service can start

systemctl start slurm
systemctl status slurm
systemctl enable slurm

 

 

Testing SLURM

 

scontrol show daemons
srun --ntasks=16 --label /bin/hostname
sbatch # submit script
salloc # create job alloc and start shell, interactive
srun # create job alloc and launch job step, MPI
sattach #
sinfo
sinfo --Node
sinfo -p debug
squeue -i60
squeue -u dyzio -t all
squeue -s -p debug
smap
sview
scontrol show partition
scontrol update PartitionName=debug MaxTime=60
scontrol show config
sacct -u dyzio
sacct -p debug
sstat
sreport
sacctmgr
sprio
sshare
sdiag
scancel --user=dyzio --state=pending
scancel 444445
strigger
# Submit a job array with index values between 0 and 31
sbatch --array=0-31 -N1 tmp
# Submit a job array with index values of 1, 3, 5 and 7
sbatch --array=1,3,5,7 -N1 tmp
# Submit a job array with index values between 1 and 7
# with a step size of 2 (i.e. 1, 3, 5 and 7)
sbatch --array=1-7:2 -N1 tmp

 

 

Any troubles?

Checkout /var/log/messages /var/log/slurm/slurmdbd.log and output from

systemctl status slurm slurmdbd munge -l

That should get you started. Drop a comment below if it did.