How to automatically backup ALL mysql databases into sql statement?

Question

MySQL Administrator> Backup Project. It is a great tool to allow you to select databases, and schedule it.

However my issue is:

Today got a few new websites (new database created) Tomorrow got a few more new websites (new databases created)

In this case, I have to always go into Backup Project> Select those remaining schema not in the backup list...

How to make it auto check for any new databases and include in the backup schedule list?

In other word, how to automate the backup of all the mysql databases (so that we don't need to worry when there's new databases created everyday).

Any way to make it happen?

I'm using Windows 2008 server, which is not sh friendly.

Answer 1

The problem with the three answers posted so far is that they do not enable you to selectively restore the databases. This can be a real problem in all but a catastrophe.

Ideally, you should have a daily backup, with some history. It should be bulletproof (--force), it should be logged (>> ...log), it should be compressed (| gzip), it should keep separate copies of each database, and it should automatically pick up any databases that are added.

Consider, rather, a shell script like this:

#!/bin/bash

Host=server.domain.com
BDir=/home/backup/backup/mysql

Dump="/usr/bin/mysqldump --skip-extended-insert --force"
MySQL=/usr/bin/mysql

Today=$(date "+%a")

# Get a list of all databases
Databases=$(echo "SHOW DATABASES" | $MySQL -h $Host)


for db in $Databases; do
        date=`date`
        file="$BDir/$Host-$db-$Today.sql.gz"
        echo "Backing up '$db' from '$Host' on '$date' to: "
        echo "   $file"
        $Dump -h $Host $db | gzip > $file
done

Which is assuming that you have a file ~/.my.cnf (chmod 600), that has:

[client]
user = "BACKUP"
password = "SOMEPASS8342783492"

Make sure that whatever user you are using for BACKUP has this grant statement:

GRANT 
  SELECT, SHOW VIEW ON *.* 
  TO BACKUP@localhost 
  IDENTIFIED BY 'SOMEPASS8342783492';

So simply add this to a nightly cronjob, and you have a daily backup that rotates each 7 days week.

0 3 * * *   backup-mysql >> backup-mysql.log 2>> backup-mysql.log

---

The backup directory then contains:

-rw-r--r-- 1 backup backup 2217482184 Sep  3 13:35 base.appcove.net-VOS4_0-20090903.sql.gz
-rw-rw-r-- 1 backup backup 2505876287 Dec 25 00:48 base.appcove.net-VOS4_0-Fri.sql.gz
-rw-r--r-- 1 backup backup 2500384029 Dec 21 00:48 base.appcove.net-VOS4_0-Mon.sql.gz
-rw-r--r-- 1 backup backup 2506849331 Dec 26 00:48 base.appcove.net-VOS4_0-Sat.sql.gz
-rw-r--r-- 1 backup backup 2499859469 Dec 20 00:48 base.appcove.net-VOS4_0-Sun.sql.gz
-rw-rw-r-- 1 backup backup 2505046147 Dec 24 00:48 base.appcove.net-VOS4_0-Thu.sql.gz
-rw-rw-r-- 1 backup backup 2502277743 Dec 22 00:48 base.appcove.net-VOS4_0-Tue.sql.gz
-rw-r--r-- 1 backup backup 2504169910 Dec 23 00:48 base.appcove.net-VOS4_0-Wed.sql.gz
-rw-r--r-- 1 backup backup   76983829 Dec 25 00:49 base.appcove.net-VOS4_Mail_0-Fri.sql.gz
-rw-r--r-- 1 backup backup   76983829 Dec 21 00:49 base.appcove.net-VOS4_Mail_0-Mon.sql.gz
-rw-r--r-- 1 backup backup   76983829 Dec 26 00:49 base.appcove.net-VOS4_Mail_0-Sat.sql.gz
-rw-r--r-- 1 backup backup   76983829 Dec 20 00:48 base.appcove.net-VOS4_Mail_0-Sun.sql.gz
-rw-rw-r-- 1 backup backup   76983829 Dec 24 00:49 base.appcove.net-VOS4_Mail_0-Thu.sql.gz
-rw-rw-r-- 1 backup backup   76983829 Dec 22 00:49 base.appcove.net-VOS4_Mail_0-Tue.sql.gz
-rw-r--r-- 1 backup backup   76983829 Dec 23 00:49 base.appcove.net-VOS4_Mail_0-Wed.sql.gz
-rw-r--r-- 1 backup backup  304803726 Dec 25 00:49 base.appcove.net-WeSell_0-Fri.sql.gz
-rw-r--r-- 1 backup backup  303480087 Dec 21 00:49 base.appcove.net-WeSell_0-Mon.sql.gz
-rw-r--r-- 1 backup backup  304710121 Dec 26 00:49 base.appcove.net-WeSell_0-Sat.sql.gz
-rw-r--r-- 1 backup backup  303791294 Dec 20 00:49 base.appcove.net-WeSell_0-Sun.sql.gz
-rw-rw-r-- 1 backup backup  305315415 Dec 24 00:49 base.appcove.net-WeSell_0-Thu.sql.gz
-rw-rw-r-- 1 backup backup  302516217 Dec 22 00:49 base.appcove.net-WeSell_0-Tue.sql.gz
-rw-r--r-- 1 backup backup  303314217 Dec 23 00:49 base.appcove.net-WeSell_0-Wed.sql.gz
-rw-r--r-- 1 backup backup     135301 Dec 25 00:30 dc40.appcove.net-mysql-Fri.sql.gz
-rw-r--r-- 1 backup backup     135301 Dec 21 00:30 dc40.appcove.net-mysql-Mon.sql.gz
-rw-r--r-- 1 backup backup     135301 Dec 26 00:30 dc40.appcove.net-mysql-Sat.sql.gz
-rw-r--r-- 1 backup backup     135301 Dec 20 00:30 dc40.appcove.net-mysql-Sun.sql.gz
-rw-rw-r-- 1 backup backup     135301 Dec 24 00:30 dc40.appcove.net-mysql-Thu.sql.gz
-rw-rw-r-- 1 backup backup     135301 Dec 22 00:30 dc40.appcove.net-mysql-Tue.sql.gz
-rw-r--r-- 1 backup backup     135301 Dec 23 00:30 dc40.appcove.net-mysql-Wed.sql.gz

Answer 2

MySQL Administrator

With MySQL Administrator you need to manually add databases to the backup.

mysqldump and --all-databases

If you approach this through a dump-command, you can use the --all-databases option to include all databases into the dump.

Answer 3

mysqldump -u <username> -p<password> --all-databases > database_backup.sql

Answer 4

Late answer, but as simple as it gets:

1. You need to change lines 3, 4 and 5 to reflect your MySQL user, password and dir where you want to store the dumps.
2. Every time it runs, it’ll delete all the previous backups (If you don’t want this, just comment #rm "$OUTPUT/*gz" > /dev/null 2>&1)

MySqlBackup.sh

#!/bin/bash
 
USER="your_user"
PASSWORD="your_password"
OUTPUT="/path/to/backup/dir"
 
rm "$OUTPUT/*gz" > /dev/null 2>&1
 
databases=`mysql --user=$USER --password=$PASSWORD -e "SHOW DATABASES;" | tr -d "| " | grep -v Database`
 
for db in $databases; do
    if [[ "$db" != "information_schema" ]] ; then
        echo "Dumping database: $db"
        mysqldump --force --opt --user=$USER --password=$PASSWORD --databases $db > $OUTPUT/`date +%Y%m%d`.$db.sql
        gzip $OUTPUT/`date +%Y%m%d`.$db.sql
    fi
done

---

---

Make it executable and run it:

chmod 700 MySqlBackup.sh
./MySqlBackup.sh

If needed, add it to the crontab so it’ll run automagically:

crontab -e
00 02 * * * /path/to/MySqlBackup.sh

In this case, it’ll run every day at 2 AM. You can learn more about crontab here.

src

Answer 5

I've been using http://sourceforge.net/projects/automysqlbackup/ to backup my MySQL databases for a couple years now and it's worked very well for me. Here's the description from the sourceforge page:

A script to take daily, weekly and monthly backups of your MySQL databases using mysqldump. Features - Backup mutiple databases - Single backup file or to a seperate file for each DB - Compress backup files - Backup remote servers - E-mail logs - More..

You can set the script to backup all databases so you don't have to change the script when new dbs are added. You can also tell it which dbs to exclude if you have a database or two that you don't want to backup for some reason.

It's well documented and there are lots of options that you can set that will cover most basic db backup needs. Since it's a single bash script, it's also easy to modify/tweak if something's not quite how you want it to be.

Answer 6

Try using

mysqldump --all-databases

Answer 7

Just my two cents. Upgrading the excelent script from @gahooa

It adds grant save. Usefull if you want to save permissions and a few of improvements and value checking. Also usage of user and pass if provided.

Hope it helps.

UDATE: Added sha1 sums for each file

#!/bin/bash
# 
# Script to backup database and grants from mysql 
# Author: Gonzalo Aguilar Delgado <gaguilar indomain level2crm.com> 
# Usage:
#       backup-mysql.sh <HOST> username password   <--- Yes this is insecure as this will get on history
# Based on script http://stackoverflow.com/questions/1963884/how-to-automatically-backup-all-mysql-databases-into-sql-statement
# from gahooa (http://stackoverflow.com/users/64004/gahooa)
#


if [ "$#" -lt 1 ]; then
    FILENAME="$(basename $0)"
    echo "Usage $FILENAME <HOST> [user] [pass]"
    exit 1
fi

#http://stackoverflow.com/questions/5947742/how-to-change-the-output-color-of-echo-in-linux
COLOR_GREEN='\e[0;32m'
COLOR_RED='\e[0;31m'
COLOR_BLACK='\e[0;30m'
COLOR_DISABLE='\e[0m'

HOST=$1

if [ ! -z "$2" ]; then 
    USER=$2
fi
PASS=$3
BACKUP_DIRECTORY="/home/gaguilar/mysql-backup"

if [ ! -x $BACKUP_DIRECTORY ]; then
    echo "Cannot access to $BACKUP_DIRECTORY"
    exit 1
fi

DUMP_COMMAND="/usr/bin/mysqldump --skip-extended-insert --force"
MYSQL_COMMAND=/usr/bin/mysql

TODAY_DATE=$(date "+%F")

COMMAND_ARGS="-h$HOST"

if [ ! -z "$USER" ]; then
    COMMAND_ARGS="$COMMAND_ARGS -u$USER" 
fi

if [ ! -z "$PASS" ]; then
    COMMAND_ARGS="$COMMAND_ARGS -p$PASS" 
fi


echo -e "Connecting to ${COLOR_BLACK}$HOST${COLOR_DISABLE} with user ${COLOR_BLACK}$USER${COLOR_DISABLE}..."
# Get a list of all databases
DB_LIST=$(echo "SHOW DATABASES" | $MYSQL_COMMAND $COMMAND_ARGS)

if [ "$?" -ne "0" ]; then
    echo "Cannot connect"
    exit 1
fi

echo "Starting backup"
for db in $DB_LIST; do
    if [ "$db" != "information_schema" ] && [ "$db" != "mysql" ] && [ "$db" != "performance_schema" ]; then
            date=`date`
            file="$BACKUP_DIRECTORY/$HOST-$db-$TODAY_DATE.sql.bz2"
            file_sum=$BACKUP_DIRECTORY/$HOST-$db-$TODAY_DATE.sql.bz2.sha1
            echo "Backing up '$db' from '$HOST' on '$date' to: "
            echo "   [$file]..."
            $DUMP_COMMAND $COMMAND_ARGS $db | bzip2 > $file
            sha1sum $file > $file_sum
            if [ "$?" -ne "0" ]; then
                    echo "Backup of '$db' from '$HOST' ${COLOR_RED}failed${COLOR_DISABLE}..."
                fi
    else
                echo "Skipping $db database because mysql internal"
    fi
done
echo -e "${COLOR_GREEN}Done backup databases${COLOR_DISABLE}"

echo "Backing up privileges of users"
USER_LIST=$(echo "SELECT CONCAT('\'',user,'\'@\'',host,'\'') FROM mysql.user WHERE user != 'root'" | $MYSQL_COMMAND -BN $COMMAND_ARGS)
grants_file="$BACKUP_DIRECTORY/$HOST-grants-$TODAY_DATE.sql"

echo "-- GRANTS FOR $HOST at $TODAY_DATE" > $grants_file
for username in $USER_LIST; do
    echo "Backing up $username"
    GRANT_LIST=$(echo "show grants for $username" | $MYSQL_COMMAND -BN $COMMAND_ARGS | sed 's/$/;/; s/\\\\/\\/g';) # 
    echo "" >> $grants_file
    echo "-- " >> $grants_file
    echo "-- BACKUP OF USERNAME [$username]" >> $grants_file
    echo "-- " >> $grants_file
    echo "$GRANT_LIST" >> $grants_file
done

echo -e "${COLOR_GREEN}Done backup users${COLOR_DISABLE}"

The output of the grant file is like this one:

-- GRANTS FOR localhost at 2014-10-15

-- 
-- BACKUP OF USERNAME ['java'@'%']
-- 
GRANT USAGE ON *.* TO 'java'@'%' IDENTIFIED BY PASSWORD 'XXX';
GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, REFERENCES, INDEX, ALTER, CREATE TEMPORARY TABLES, LOCK TABLES, EXECUTE, CREATE VIEW, SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE, TRIGGER ON `widgetdb`.* TO 'java'@'%';

-- 
-- BACKUP OF USERNAME ['planes2'@'%']
-- 
GRANT USAGE ON *.* TO 'planes2'@'%' IDENTIFIED BY PASSWORD 'XXX';
GRANT ALL PRIVILEGES ON `planes2`.* TO 'planes2'@'%';

Answer 8

I made a script , which will take all database backup expect information_schema and performance_schema.

Will create date and time wise folder , save database its db folder name wise and gunzip . I created it today and found working 100% ok

USER=root
PASSWORD='password'

#Place where you want to preserved backup.
OUTPUT="."

TIMESTAMP=`date +%Y-%m-%d_%H-%M-%S`;
mkdir $OUTPUT/$TIMESTAMP;
cd $OUTPUT/$TIMESTAMP;
echo "Starting MySQL Backup";
echo 'date';
databases=$(mysql -u $USER -p$PASSWORD -e "SHOW DATABASES;" | tr -d "| " | grep -Ev "(Database|information_schema|performance_schema)");

echo $databases;


for db in $databases; do
        mysqldump --force --opt -u $USER -p$PASSWORD --databases $db > $OUTPUT/dbbackup-$TIMESTAMP-$db.sql
        gzip $OUTPUT/dbbackup-$TIMESTAMP-$db.sql
done
echo "Finished MySQL Backup";
echo 'date';

Answer 9

To ignore the errors use --force option

mysqldump -u root --force --all-databases > d:\all.sql