分析MariaDB初始化脚本mysql_install_db

在初始化MySQL的过程中经常会碰到各种问题，如

FATAL ERROR: Could not find ./bin/my_print_defaults

ERROR: 1 Can't create/write to file '/root/test/data/mysql/db.MYI' (Errcode: 13 - Permission denied)

出现这些问题的原因无非是两种，

1> 没有传递合适的参数。

譬如：

[root@localhost ~]# /usr/test/mariadb-10.1.16-linux-x86_64/scripts/mysql_install_db --datadir=/usr/test/mariadb-10.1.16-linux-x86_64/ --user=mysql &

没有指定basedir，无法找到my_print_defaults命令

2> 目录对当前用户没有权限

譬如：

[root@localhost test]# /usr/test/Percona-Server-5.6.31-rel77.0-Linux.x86_64.ssl101/scripts/mysql_install_db --basedir=/usr/test/Percona-Server-5.6.31-rel77.0-Linux.x86_64.ssl101/ --user=mysql

没有指定datadir，默认是当前目录下的data目录。如果放到/usr下执行倒是没有问题，/root目录本身的权限是500的。对于其它用户，它是没有权限在/root目录及其子目录创建文件的。

下面对MariaDB的初始化脚本进行较为详细的解析。

首先定义初始化变量

这里面ldata即数据目录，如果在执行mysql_install_db脚本时，没有显示指定--datadir，则datadir默认在当前目录下的data。

basedir=""builddir=""ldata="./data"langdir=""srcdir=""args=""defaults=""mysqld_opt=""user=""force=0in_rpm=0ip_only=0cross_bootstrap=0

定义usage函数

在使用mysql_install_db --help即输出的是usage函数的内容

usage(){ cat <<EOFUsage: $0 [OPTIONS] --basedir=path The path to the MariaDB installation directory. --builddir=path If using --srcdir with out-of-directory builds, you will need to set this to the location of the build directory where built files reside. --cross-bootstrap For internal use. Used when building the MariaDB system tables on a different host than the target. --datadir=path The path to the MariaDB data directory. --defaults-extra-file=name Read this file after the global files are read. --defaults-file=name Only read default options from the given file name. --force Causes mysql_install_db to run even if DNS does not work. In that case, grant table entries that normally use hostnames will use IP addresses. --help Display this help and exit. --ldata=path The path to the MariaDB data directory. Same as --datadir. --no-defaults Don't read default options from any option file. --defaults-file=path Read only this configuration file. --rpm For internal use. This option is used by RPM files during the MariaDB installation process. --skip-name-resolve Use IP addresses rather than hostnames when creating grant table entries. This option can be useful if your DNS does not work. --srcdir=path The path to the MariaDB source directory. This option uses the compiled binaries and support files within the source tree, useful for if you don't want to install MariaDB yet and just want to create the system tables. --user=user_name The login username to use for running mysqld. Files and directories created by mysqld will be owned by this user. You must be root to use this option. By default mysqld runs using your current login name and files and directories that it creates will be owned by you.All other options are passed to the mysqld programEOF exit 1}

定义打印函数

s_echo(){ if test "$in_rpm" -eq 0 -a "$cross_bootstrap" -eq 0 then  echo "$1" fi}

其中$in_rpm对应--rpm参数，$cross_bootstrap对应 --cross-bootstrap参数。

关于这两个参数的作用，可以参考上面usage函数中的说明。

在脚本执行失败时，调用的函数

link_to_help(){ echo echo "The latest information about mysql_install_db is available at" echo "https://mariadb.com/kb/en/installing-system-tables-mysql_install_db"}

定义参数解析函数

parse_arg(){ echo "$1" | sed -e 's/^[^=]*=//'}

其中s是替换，^代表行首定位符，[]代表匹配一组字符里的任意字符，*匹配0个或多个前一字符，[^ ]代表匹配不在指定范围内的字符。

这个函数实现的效果是截取“=”号后的字符。

譬如，输入的变量是--basedir=/usr/test，则输出的结果是/usr/test

定义命令行解析函数

parse_arguments(){ # We only need to pass arguments through to the server if we don't # handle them here. So, we collect unrecognized options (passed on # the command line) into the args variable. pick_args= if test "$1" = PICK-ARGS-FROM-ARGV then  pick_args=1  shift fi for arg do  case "$arg" in   --force) force=1 ;;   --basedir=*) basedir=`parse_arg "$arg"` ;;   --builddir=*) builddir=`parse_arg "$arg"` ;;   --srcdir=*) srcdir=`parse_arg "$arg"` ;;   --ldata=*|--datadir=*|--data=*) ldata=`parse_arg "$arg"` ;;   --user=*)    # Note that the user will be passed to mysqld so that it runs    # as 'user' (crucial e.g. if log-bin=/some_other_path/    # where a chown of datadir won't help)    user=`parse_arg "$arg"` ;;   --skip-name-resolve) ip_only=1 ;;   --verbose) verbose=1 ;; # Obsolete   --rpm) in_rpm=1 ;;   --help) usage ;;   --no-defaults|--defaults-file=*|--defaults-extra-file=*)    defaults="$arg" ;;   --cross-bootstrap|--windows)    # Used when building the MariaDB system tables on a different host than    # the target. The platform-independent files that are created in    # --datadir on the host can be copied to the target system.    #    # The most common use for this feature is in the Windows installer    # which will take the files from datadir and include them as part of    # the install package. See top-level 'dist-hook' make target.    #    # --windows is a deprecated alias    cross_bootstrap=1 ;;   *)    if test -n "$pick_args"    then     # This sed command makes sure that any special chars are quoted,     # so the arg gets passed exactly to the server.     # XXX: This is broken; true fix requires using eval and proper     # quoting of every single arg ($basedir, $ldata, etc.)     #args="$args "`echo "$arg" | sed -e 's,\([^a-zA-Z0-9_.-]\),\\\\\1,g'`     args="$args $arg"    fi    ;;  esac done}

其中，for arg相当于for arg in '"$@"'。

PS：

关于shell脚本中$*，$@和$#三者的区别：

脚本名称叫test.sh 传入三个参数: 1 2 3

运行test.sh 1 2 3后

$*为"1 2 3"（一起被引号包住）

$@为"1" "2" "3"（分别被包住）

$#为3（参数数量）

判断给定的文件是否存在

find_in_basedir(){ case "$1" in  --dir)   return_dir=1; shift   ;; esac file=$1; shift for dir in "$@" do  if test -f "$basedir/$dir/$file"  then   if test -n "$return_dir"   then    echo "$basedir/$dir"   else    echo "$basedir/$dir/$file"   fi   break  fi done}

其中 test -f是判断给定的文件是否存在，test -n代表给定的变量是否为非空。

解析命令行参数

正如它注释中提到的，第一次解析的参数的目录在于得到my_print_defaults命令的路径。

# Ok, let's go. We first need to parse arguments which are required by# my_print_defaults so that we can execute it first, then later re-parse# the command line to add any extra bits that we need.parse_arguments "$@"

确认my_print_defaults命令的路径

## We can now find my_print_defaults. This script supports:##  --srcdir=path pointing to compiled source tree#  --basedir=path pointing to installed binary location## or default to compiled-in locations.#if test -n "$srcdir" && test -n "$basedir"then echo "ERROR: Specify either --basedir or --srcdir, not both." link_to_help exit 1fiif test -n "$srcdir"then if test -z "$builddir" then  builddir="$srcdir" fi print_defaults="$builddir/extra/my_print_defaults"elif test -n "$basedir"then print_defaults=`find_in_basedir my_print_defaults bin extra` if test -z "$print_defaults" then  cannot_find_file my_print_defaults $basedir/bin $basedir/extra  exit 1 fielse print_defaults="./bin/my_print_defaults"fi

首先，判断srcdir和basedir是否指定，如果两者都指定的话，则会报错，在这里，srcdir指向的是源码包的编译路径，basedir指向的是二进制包的解压路径。毕竟是两种不同的安装方式，一个是源码安装，一个是二进制压缩包直接解压，两者的目录层次会不一样。

上述脚本的判断逻辑是，如果$srcdir指定了，则my_print_defaults命令位于$builddir/extra中。

如果没有指定$srcdir，而指定了$basedir，则my_print_defaults要么位于$basedir/bin中，要么位于$basedir/extra中。如果没有找到该命令，则直接报错退出脚本。

如果$srcdir和$basedir都没有指定，则my_print_defaults默认在当前目录中的bin目录下，此时，它假定你是在$basedir上执行该初始化命令的。

判断my_print_defaults对于当前用户是否有可执行权限

if test ! -x "$print_defaults"then cannot_find_file "$print_defaults" exit 1fi

获取配置文件中[mysqld]和[mysql_install_db]区域的值

# Now we can get arguments from the groups [mysqld] and [mysql_install_db]# in the my.cfg file, then re-run to merge with command line arguments.parse_arguments `"$print_defaults" $defaults --mysqld mysql_install_db`

既然能执行my_print_defaults，则可以根据该命令得到配置文件中[mysqld]和[mysql_install_db]区域的值

上面的$defaults是之前定义的--no-defaults，--defaults-file，--defaults-extra-file

譬如如果--defaults-file=/usr/local/mysql，则defaults="--defaults-file=/usr/local/mysql"。

对于下面这个命令"$print_defaults" $defaults --mysqld mysql_install_db，针对于我本机的环境，它实际上执行的是

# /usr/test/mariadb-10.1.16-linux-x86_64/bin/my_print_defaults --defaults-file=/usr/test/mariadb-10.1.16-linux-x86_64/my.cnf --mysqld mysql_install_db

--port=3308
--basedir=/usr/test/mariadb-10.1.16-linux-x86_64
--datadir=/usr/test/mariadb-10.1.16-linux-x86_64/data
--skip-external-locking
--user=mysql
--key_buffer_size=16K
--max_allowed_packet=1M
--table_open_cache=4
--sort_buffer_size=64K
--read_buffer_size=256K
--read_rnd_buffer_size=256K
--net_buffer_length=2K
--thread_stack=240K
--server-id=1

其中，--mysqld前面不用带“--”也行。

但是5.6.31 MySQL社区版的my_print_defaults只支持不带“--”的mysqld，由此可见，两者在语法方面还是有一定的差别的。

根据上面的输出的结果，则脚本中的这个命令等价于

parse_arguments --port=3308 --basedir=/usr/test/mariadb-10.1.16-linux-x86_64 --datadir=/usr/test/mariadb-10.1.16-linux-x86_64/data --skip-external-locking --user=mysql --key_buffer_size=16K --max_allowed_packet=1M --table_open_cache=4 --sort_buffer_size=64K --read_buffer_size=256K --read_rnd_buffer_size=256K --net_buffer_length=2K --thread_stack=240K --server-id=1

执行完上面这个命令后，实际上只有三个参数传递进来了，--basedir，--datadir，--user

再次执行命令解析函数

parse_arguments PICK-ARGS-FROM-ARGV "$@"

这次调用的目的是捕捉命令行中传递的其它参数。

譬如执行如下命令

# /usr/test/mariadb-10.1.16-linux-x86_64/scripts/mysql_install_db --defaults-file=/usr/test/mariadb-10.1.16-linux-x86_64/my.cnf --basedir=/usr/test/mariadb-10.1.16-linux-x86_64/ 123 456 789

则args中的值为' 123 456 789'

定义初始化所需文件的路径

# Configure paths to support filesif test -n "$srcdir"then basedir="$builddir" bindir="$basedir/client" extra_bindir="$basedir/extra" mysqld="$basedir/sql/mysqld" langdir="$basedir/sql/share/english" pkgdatadir="$srcdir/scripts" scriptdir="$srcdir/scripts"elif test -n "$basedir"then bindir="$basedir/bin" extra_bindir="$bindir" mysqld=`find_in_basedir mysqld libexec sbin bin` if test -z "$mysqld" then  cannot_find_file mysqld $basedir/libexec $basedir/sbin $basedir/bin  exit 1 fi langdir=`find_in_basedir --dir errmsg.sys share/english share/mysql/english` if test -z "$langdir" then  cannot_find_file errmsg.sys $basedir/share/english $basedir/share/mysql/english  exit 1 fi pkgdatadir=`find_in_basedir --dir fill_help_tables.sql share share/mysql` if test -z "$pkgdatadir" then  cannot_find_file fill_help_tables.sql $basedir/share $basedir/share/mysql  exit 1 fi scriptdir="$basedir/scripts"else basedir="." bindir="./bin" extra_bindir="$bindir" mysqld="./bin/mysqld" pkgdatadir="./share" scriptdir="./bin"fi

也是分三种情况，即指定了srcdir，指定了basedir，或者两者都没有指定。

在这里，说说第二种和第三种情况

若指定了basedir，则会定义三个路径

1> mysqld，mysqld一般会存放在如下三个路径中，$basedir/libexec/mysqld，$basedir/sbin/mysqld，$basedir/bin/mysqld

2> langdir，该路径是errmsg.sys的存放路径，该文件与mysql的错误代码有关。升级时该文件即需要更新

3> pkgdatadir，该路径存放mysql库，performance_schema库的创建脚本，不仅仅是fill_help_tables.sql。

如果没有指定srcdir和basedir，则默认将当前路径设置为basedir。

定义数据库创建脚本的路径

在上面路径确认好的情况下，进一步确认脚本是否存在，mysqld是否有可执行权限，确认errmsg.sys是否存在

# Set up paths to SQL scripts required for bootstrapfill_help_tables="$pkgdatadir/fill_help_tables.sql"create_system_tables="$pkgdatadir/mysql_system_tables.sql"create_system_tables2="$pkgdatadir/mysql_performance_tables.sql"fill_system_tables="$pkgdatadir/mysql_system_tables_data.sql"maria_add_gis_sp="$pkgdatadir/maria_add_gis_sp_bootstrap.sql"for f in "$fill_help_tables" "$create_system_tables" "$create_system_tables2" "$fill_system_tables" "$maria_add_gis_sp"do if test ! -f "$f" then  cannot_find_file "$f"  exit 1 fidoneif test ! -x "$mysqld"then cannot_find_file "$mysqld" exit 1fiif test -n "$langdir"then if test ! -f "$langdir/errmsg.sys" then  cannot_find_file "$langdir/errmsg.sys"  exit 1 fi mysqld_opt="--lc-messages-dir=$langdir/.."else mysqld_opt="--lc-messages=en_US"fi

确认主机名以及主机名是否有效

它的判断逻辑是如果--cross-bootstrap，--rpm，--force没有显式指定的话，则查看主机名是否能被解析成ip。

首先解析的是主机名，如果没有解析成功，则解析localhost

# Try to determine the hostnamehostname=`hostname`# Check if hostname is validif test "$cross_bootstrap" -eq 0 -a "$in_rpm" -eq 0 -a "$force" -eq 0then resolved=`"$extra_bindir/resolveip" $hostname 2>&1` if test $? -ne 0 then  resolved=`"$extra_bindir/resolveip" localhost 2>&1`  if test $? -ne 0  then   echo "Neither host '$hostname' nor 'localhost' could be looked up with"   echo "'$extra_bindir/resolveip'"   echo "Please configure the 'hostname' command to return a correct"   echo "hostname."   echo "If you want to solve this at a later stage, restart this script"   echo "with the --force option"   link_to_help   exit 1  fi  echo "WARNING: The host '$hostname' could not be looked up with resolveip."  echo "This probably means that your libc libraries are not 100 % compatible"  echo "with this binary MariaDB version. The MariaDB daemon, mysqld, should work"  echo "normally with the exception that host name resolving will not work."  echo "This means that you should use IP addresses instead of hostnames"  echo "when specifying MariaDB privileges !" fifi

如果指定了--skip-name-resolve参数，则将主机名解析为IP

if test "$ip_only" -eq 1then hostname=`echo "$resolved" | awk '/ /{print $6}'`fi

创建数据目录

# Create database directoriesfor dir in "$ldata" "$ldata/mysql" "$ldata/test"do if test ! -d "$dir" then  if ! `mkdir -p "$dir"`  then   echo "Fatal error Can't create database directory '$dir'"   link_to_help   exit 1  fi  chmod 700 "$dir" fi if test -n "$user" then  chown $user "$dir"  if test $? -ne 0  then   echo "Cannot change ownership of the database directories to the '$user'"   echo "user. Check that you have the necessary permissions and try again."   exit 1  fi fidone

可以看到，会创建三个目录，datadir，以及datadir下的mysql目录和test目录，这个对应mysql库和test库。

如果目录不存在，则创建，并将目录权限设置为700，如果指定了--user参数，则将目录的属主修改为指定的用户

如果指定了--user参数，则添加到之前的参数列表中

if test -n "$user"then args="$args --user=$user"fi

--cross-bootstrap参数是用于跨平台启动的，具体可以参考上面parse_arguments函数中对该参数的解释

如果指定了该参数，则会过滤初始化脚本中有关当前主机名的设置。

# When doing a "cross bootstrap" install, no reference to the current# host should be added to the system tables. So we filter out any# lines which contain the current host name.if test $cross_bootstrap -eq 1then filter_cmd_line="sed -e '/@current_hostname/d'"else filter_cmd_line="cat"fi

配置mysqld命令行

感觉MYSQLD_BOOTSTRAP变量出现得莫名其妙，上文中也没给出任何定义

这样执行的效果是如果定义了MYSQLD_BOOTSTRAP，则$MYSQLD_BOOTSTRAP值赋给mysqld_bootstrap，如果没有定义，则$mysqld的值赋给mysqld_bootstrap

# Configure mysqld command linemysqld_bootstrap="${MYSQLD_BOOTSTRAP-$mysqld}"mysqld_install_cmd_line(){ "$mysqld_bootstrap" $defaults "$mysqld_opt" --bootstrap \ "--basedir=$basedir" "--datadir=$ldata" --log-warnings=0 --enforce-storage-engine="" \ $args --max_allowed_packet=8M \ --net_buffer_length=16K}

若以如下方式初始化mysql

# /usr/test/mariadb-10.1.16-linux-x86_64/scripts/mysql_install_db --defaults-file=/usr/test/mariadb-10.1.16-linux-x86_64/my.cnf --basedir=/usr/test/mariadb-10.1.16-linux-x86_64/

则上面这个函数相当于

/usr/test/mariadb-10.1.16-linux-x86_64/bin/mysqld --defaults-file=/usr/test/mariadb-10.1.16-linux-x86_64/my.cnf --lc-messages-dir=/usr/test/mariadb-10.1.16-linux-x86_64/share --bootstrap --basedir=/usr/test/mariadb-10.1.16-linux-x86_64/ --datadir=/usr/test/mariadb-10.1.16-linux-x86_64/data --log-warnings=0 --enforce-storage-engine="" --user=mysql --max_allowed_packet=8M --net_buffer_length=16K

执行数据库初始化脚本

分为三部分

1> 系统表

2> fill_help_tables.sql，该文件用于生成help contents的内容

3> OpenGIS

其中，--bootstrap代表Used by mysql installation scripts。

虽然同样是执行的mysqld命令，但因为指定了--bootstrap参数，只是执行了数据库初始化脚本中的命令，并没有启动数据库。

# Create the system and help tables by passing them to "mysqld --bootstrap"s_echo "Installing MariaDB/MySQL system tables in '$ldata' ..."if { echo "use mysql;"; cat "$create_system_tables" "$create_system_tables2" "$fill_system_tables"; } | eval "$filter_cmd_line" | mysqld_install_cmd_line > /dev/nullthen s_echo "OK"else echo echo "Installation of system tables failed! Examine the logs in" echo "$ldata for more information." echo echo "The problem could be conflicting information in an external" echo "my.cnf files. You can ignore these by doing:" echo echo "  shell> $scriptdir/scripts/mysql_install_db --defaults-file=~/.my.cnf" echo echo "You can also try to start the mysqld daemon with:" echo echo "  shell> $mysqld --skip-grant --general-log &" echo echo "and use the command line tool $bindir/mysql" echo "to connect to the mysql database and look at the grant tables:" echo echo "  shell> $bindir/mysql -u root mysql" echo "  mysql> show tables;" echo echo "Try 'mysqld --help' if you have problems with paths. Using" echo "--general-log gives you a log in $ldata that may be helpful." link_to_help echo "MariaDB is hosted on launchpad; You can find the latest source and" echo "email lists at http://launchpad.net/maria" echo echo "Please check all of the above before submitting a bug report" echo "at http://mariadb.org/jira" echo exit 1fis_echo "Filling help tables..."if { echo "use mysql;"; cat "$fill_help_tables"; } | mysqld_install_cmd_line > /dev/nullthen s_echo "OK"else echo echo "WARNING: HELP FILES ARE NOT COMPLETELY INSTALLED!" echo "The \"HELP\" command might not work properly."fis_echo "Creating OpenGIS required SP-s..."if { echo "use test;"; cat "$maria_add_gis_sp"; } | mysqld_install_cmd_line > /dev/nullthen s_echo "OK"else echo echo "WARNING: OPENGIS REQUIRED SP-S WERE NOT COMPLETELY INSTALLED!" echo "GIS extentions might not work properly."fi

输出相关信息

针对的是--cross-bootstrap，--srcdir，--rpm这三个参数。

# Don't output verbose information if running inside bootstrap or using# --srcdir for testing. In such cases, there's no end user looking at# the screen.if test "$cross_bootstrap" -eq 0 && test -z "$srcdir"then s_echo s_echo "To start mysqld at boot time you have to copy" s_echo "support-files/mysql.server to the right place for your system" echo echo "PLEASE REMEMBER TO SET A PASSWORD FOR THE MariaDB root USER !" echo "To do so, start the server, then issue the following commands:" echo echo "'$bindir/mysqladmin' -u root password 'new-password'" echo "'$bindir/mysqladmin' -u root -h $hostname password 'new-password'" echo echo "Alternatively you can run:" echo "'$bindir/mysql_secure_installation'" echo echo "which will also give you the option of removing the test" echo "databases and anonymous user created by default. This is" echo "strongly recommended for production servers." echo echo "See the MariaDB Knowledgebase at http://mariadb.com/kb or the" echo "MySQL manual for more instructions." if test "$in_rpm" -eq 0 then  echo  echo "You can start the MariaDB daemon with:"  echo "cd '$basedir' ; $bindir/mysqld_safe --datadir='$ldata'"  echo  echo "You can test the MariaDB daemon with mysql-test-run.pl"  echo "cd '$basedir/mysql-test' ; perl mysql-test-run.pl" fi echo echo "Please report any problems at http://mariadb.org/jira" echo echo "The latest information about MariaDB is available at http://mariadb.org/." echo "You can find additional information about the MySQL part at:" echo "http://dev.mysql.com" echo "Support MariaDB development by buying support/new features from MariaDB" echo "Corporation Ab. You can contact us about this at sales@mariadb.com." echo "Alternatively consider joining our community based development effort:" echo "http://mariadb.com/kb/en/contributing-to-the-mariadb-project/" echofiexit 0

总结：

一、因为MySQL社区版本以及Percona版本的mysql_install_db都是用perl写的，只有MariaDB是shell写的，个人对shell比较熟悉，所以就对MariaDB的进行分析了。毕竟，这三个版本的初始化逻辑大同小异。

二、MariaDB的初始化的流程如下（为了简化，在这里就不考虑源码编译的情况，只考虑二进制包的初始化）：

1. 首先判断传递的参数中是否有basedir，如果有，则查找my_print_defaults命令所在的路径，如果没有找到，则直接报错退出脚本。

2. 如果没有传递basedir参数，则将当前目录下的bin目录设置为my_print_defaults所在的路径。

3. 判断my_print_defaults对于当前用户是否有可执行权限，如果没有，也直接报错退出。

4. 根据my_print_defaults，查看配置文件中[mysqld]和[mysql_install_db]的内容，并通过parse_arguments参数赋给对应的变量

5. 查找mysqld的路径，如果没有找到，直接退出

6. 查看errmsg.sys的路径，如果没有找到，直接退出

7. 查看初始化SQL脚本的的路径，如果没有找到，直接退出

8. 确认mysqld是否有可执行权限

9. 设置--lc-messages-dir目录的位置