Xenserver 启动 xapi 报错 :Starting xapi: ..failed to start xapi (daemon disappeared) FAILED

Xenserver 服务器已经运行一年了,一直正常。今天早上远程到VM上面时,感觉很慢,不正常。

查看网络与I/O读写都是正常的。

这时用户也反应速度比平时慢多了。

于是检查XEN系统时:

启动 xapi 报错:

Stopping xapi: cannot stop xapi: xapi is not running. FAILED
Starting xapi: ..failed to start xapi (daemon disappeared) FAILED

怎么操作都启动不了,即使偶尔可启动,xencenter过会就自动断开。

一直花了好长时间,都没找到原因。google 资料后,有提到查看 daemon.log日志文件。

然后查看 信息如下:

Oct 25 11:44:27 xenserver-253 xapi: [error|xenserver-253|0 thread_zero||xapi] Caught exception at toplevel: 'Sys_error("No space left on device")'

Oct 25 11:44:27 xenserver-253 xapi: [ info|xenserver-253|0 thread_zero||watchdog] received exit code 2
Oct 25 11:44:27 xenserver-253 xapi: [ info|xenserver-253|0 thread_zero||watchdog] Received 2 bad exits within no-retry-interval. Giving up.
Oct 25 11:49:28 xenserver-253 v6d: [ info|xenserver-253|0||watchdog] (Re)starting v6d...
Oct 25 11:49:28 xenserver-253 xapi: [ info|xenserver-253|0 thread_zero||watchdog] (Re)starting xapi...
Oct 25 11:49:28 xenserver-253 xapi: [ warn|xenserver-253|0 thread_zero|server_init D:32a43fac18d2|startup] task [Checking control domain] exception: Sys_error("No space left on device")
Oct 25 11:49:29 xenserver-253 xapi: [error|xenserver-253|0 thread_zero|server_init D:32a43fac18d2|xapi] Caught exception at toplevel: 'Sys_error("No space left on device")'
Oct 25 11:49:30 xenserver-253 xapi: [ info|xenserver-253|0 thread_zero||watchdog] received exit code 2
Oct 25 11:49:30 xenserver-253 xapi: [ info|xenserver-253|0 thread_zero||watchdog] Received bad exit, retrying
Oct 25 11:49:30 xenserver-253 xapi: [ info|xenserver-253|0 thread_zero||watchdog] (Re)starting xapi...
Oct 25 11:49:30 xenserver-253 xapi: [ warn|xenserver-253|0 thread_zero|server_init D:e6f840e7f54c|startup] task [Initing stunnel path] exception: Sys_error("No space left on device")
Oct 25 11:49:31 xenserver-253 xapi: [error|xenserver-253|0 thread_zero|server_init D:e6f840e7f54c|xapi] Caught exception at toplevel: 'Sys_error("No space left on device")'
Oct 25 11:49:31 xenserver-253 xapi: [ info|xenserver-253|0 thread_zero||watchdog] received exit code 2

Oct 25 11:49:31 xenserver-253 xapi: [ info|xenserver-253|0 thread_zero||watchdog] Received 2 bad exits within no-retry-interval. Giving up.

无意中发现两个报错比较特别:

Oct 25 11:49:30 xenserver-253 xapi: [ warn|xenserver-253|0 thread_zero|server_init D:e6f840e7f54c|startup] task [Initing stunnel path] exception: Sys_error("No space left on device")
Oct 25 11:49:31 xenserver-253 xapi: [error|xenserver-253|0 thread_zero|server_init D:e6f840e7f54c|xapi] Caught exception at toplevel: 'Sys_error("No space left on device")'

很明显提示说 硬盘分区空间不足.

[root@xenserver-253 log]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1             4.0G  2.1G   0      100% /
none                  380M     0  380M   0% /dev/shm
/opt/xensource/packages/iso/XenCenter.iso
                       44M   44M     0 100% /var/xen/xc-install
//10.10.254.16/iso    4.0G  2.0G  2.1G  50% /var/run/sr-mount/fbfa8adf-62b9-d870-c361-adb3e48a1682

果然,空间使用 100%了。

然后删除下日志文件占用空间多的。

# echo "">daemon.log

# echo "">audit.log

然后再查看空间使用,恢复正常.

[root@xenserver-253 log]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1             4.0G  2.1G  1.7G  55% /
none                  380M     0  380M   0% /dev/shm
/opt/xensource/packages/iso/XenCenter.iso
                       44M   44M     0 100% /var/xen/xc-install
//10.10.254.16/iso    4.0G  2.0G  2.1G  50% /var/run/sr-mount/fbfa8adf-62b9-d870-c361-adb3e48a1682