Tag: development server

Munin: This RRD was created on another architecture – migrating between 32 and 64bit systems

When I was moving Senpuu.net to a new location, I've wanted to preserve my Munin stats (since my new server have almost identical configuration they will be very useful to me). Unfortunately moving RRD files between different architectures is not possible. To copy Munin data to a new server with a different architecture, we need to use rrdtool (dump and restore).

# Error example from /var/log/munin/munin-node.log
This RRD was created on another architecture

rrd dump and restore

Lets quote a man for rrdtool:

Dump:

Dump the contents of an RRD in plain ASCII. In connection with restore you can use this to move an RRD from one computer architecture to another.

Restore:

Restore an RRD in XML format to a binary RRD.

Now everything looks better. We can easily dump RRD files to XML, move them to the other server and then easily restore them to RRD format.

To dump all RRD files for given node, you need to get (as root) to RRD files dir (by default it should be somewhere near /var/lib/munin) and execute following code:

cd /var/lib/munin/senpuu
# Create a dir for RRD dumps
mkdir ./../rrd_dump
 for i in ./*.rrd;do rrdtool dump $i ./../rrd_dump/$i.xml;done

The above code will create XML dumps in appropriate directory. When this is finished, you need to download all the files and upload them to your new server. Then execute:

# Execute this on a new server
cd /var/lib/munin/rrd_dump
for i in ./*.xml; do rrdtool restore "$i" "../senpuu/${i%.xml}"; done

after this you might reset munin-node just to be sure that everything will work fine:

/etc/init.d/munin-node restart

Now you should have your old charts up and running.

Slowing down (limiting) tar, mysqldump or other processes to save IO bandwidth

Sometimes we want to perform some sort of tasks that consume whole available IO bandwidth. This may lead to some unexpected behaviours from our OS. OS might even kill the given process due to resource lack.

Lets take an example. We want to tar.gz a huge directory with a lot of files in it. Our machine also have a web-server which serves several sites. If we start "taring" our directory, it might lead to timeouts on server (it won't be able to respond as fast as we would expect). On the other hand, we don't care so much about the time needed to create archive file. We always can throw it in screen and detach it.

# Standard approach - will slow IO response time
tar -cvf ./dir.tar ./dir

pv to the rescue!

To slow things down to a tolerable level we will use pv tool. pv allows a user to see the progress of data through a pipeline, by giving information such as time elapsed, percentage completed (with progress bar), current throughput rate, total data transferred, and ETA. It can also limit the speed of incoming data, if used wisely.

To tar a file with a given speed (in MB) we need to use following command:

tar -chf - ./dir | pv -L 2m > ./dir.tar

Above example will allow us to tar ./dir with max speed 2MB/s.

We could use the same method to slow down a mysqldump method:

mysqldump --add-drop-table -u user -p password -h host db_name | bzip2 -c > ./dump.sql.bz2

Copyright © 2024 Closer to Code

Theme by Anders NorenUp ↑