服务器内存不足致使LINUX KILL MYSQLD 故障
发布时间：2022-03-26 06:03:42 所属栏目：编程 来源：互联网
导读：环境说明 OS操作系统：SUSE Linux 11 数据库版 本：MYSQL 5.6 故障问题描述 客户反映MYSQL数据库因为内存不足被重启。 故障分析 1、根据MYSQL报警日志确认问题： 2016-10-24 12:28:49 29226 [Note] Plugin FEDERATED is disabled. 2016-10-24 12:28:49 29226
|
环境说明 OS操作系统：SUSE Linux 11 数据库版 本：MYSQL 5.6 故障问题描述 客户反映MYSQL数据库因为内存不足被重启。 故障分析 1、根据MYSQL报警日志确认问题： 2016-10-24 12:28:49 29226 [Note] Plugin 'FEDERATED' is disabled. 2016-10-24 12:28:49 29226 [Note] InnoDB: Using atomics to ref count buffer pool pages 2016-10-24 12:28:49 29226 [Note] InnoDB: The InnoDB memory heap is disabled 2016-10-24 12:28:49 29226 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2016-10-24 12:28:49 29226 [Note] InnoDB: Compressed tables use zlib 1.2.3 2016-10-24 12:28:49 29226 [Note] InnoDB: Using Linux native AIO 2016-10-24 12:28:49 29226 [Note] InnoDB: Using CPU crc32 instructions 2016-10-24 12:28:49 29226 [Note] InnoDB: Initializing buffer pool, size = 2.0G 2016-10-24 12:28:50 29226 [Note] InnoDB: Completed initialization of buffer pool 2016-10-24 12:28:50 29226 [Note] InnoDB: Highest supported file format is Barracuda. 2016-10-24 12:28:50 29226 [Note] InnoDB: Log scan progressed past the checkpoint lsn 27852365485 2016-10-24 12:28:50 29226 [Note] InnoDB: Database was not shutdown normally! 2016-10-24 12:28:50 29226 [Note] InnoDB: Starting crash recovery. 2016-10-24 12:28:50 29226 [Note] InnoDB: Reading tablespace information from the .ibd files... 2016-10-24 12:28:50 29226 [Note] InnoDB: Restoring possible half-written data pages 2016-10-24 12:28:50 29226 [Note] InnoDB: from the doublewrite buffer... InnoDB: Doing recovery: scanned up to log sequence number 27852370215 2016-10-24 12:28:51 29226 [Note] InnoDB: Starting an apply batch of log records to the database... InnoDB: Progress in percent: 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 InnoDB: Apply batch completed InnoDB: Last MySQL binlog file position 0 401322767, file name TSPTONLDB01-it_mgmad-bin.000023 2016-10-24 12:28:52 29226 [Note] InnoDB: 128 rollback segment(s) are active. 2016-10-24 12:28:52 29226 [Note] InnoDB: 5.6.19 started; log sequence number 27852370215 2016-10-24 12:28:52 29226 [Note] Recovering after a crash using /dbdata/it_mgmad/binlog/it_mgmad/TSPTONLDB01-it_mgmad-bin 2016-10-24 12:29:00 29226 [Note] Starting crash recovery... 2016-10-24 12:29:00 29226 [Note] Crash recovery finished. 2016-10-24 12:29:06 29226 [Note] Server hostname (bind-address): '*'; port: 60002 2016-10-24 12:29:06 29226 [Note] IPv6 is available. 2016-10-24 12:29:06 29226 [Note] - '::' resolves to '::'; 2016-10-24 12:29:06 29226 [Note] Server socket created on IP: '::'. 2016-10-24 12:29:06 29226 [Warning] 'proxies_priv' entry '@ root@tsptonldb01' ignored in --skip-name-resolve mode. 2016-10-24 12:29:06 29226 [Note] Event Scheduler: Loaded 0 events 2016-10-24 12:29:06 29226 [Note] /usr/local/mysql/bin/mysqld: ready for connections. Version: '5.6.19-upsql-0.1.1-log' socket: '/etc/mysock/it_mgmad.sock' port: 60002 UpSQL Server 2016-10-24 12:30:21 29226 [ERROR] /usr/local/mysql/bin/mysqld: Out of memory (Needed 8160 bytes) 2016-10-24 12:30:21 29226 [ERROR] Out of memory; check if mysqld or some other process uses all available memory; if not, you may have to use 'ulimit' to allow mysqld to use more memory or you can add more swap space 161024 12:31:35 mysqld_safe Number of processes running now: 0 161024 12:31:35 mysqld_safe mysqld restarted 2016-10-24 12:31:37 29659 [Warning] Buffered warning: Could not increase number of max_open_files to more than 10240 (request: 25000) 从上面的信息，我们看到， MSYQL报内存不足的错误无法启动。 2. 操作系统日志提示错误 Oct 25 13:41:07 TSPTONLDB01 kernel: [14317509.996482] Mem-Info: Oct 25 13:41:07 TSPTONLDB01 kernel: [14317509.996483] Node 0 DMA per-cpu: Oct 25 13:41:07 TSPTONLDB01 kernel: [14317509.996485] CPU 0: hi: 0, btch: 1 usd: 0 Oct 25 13:41:07 TSPTONLDB01 kernel: [14317509.996486] CPU 1: hi: 0, btch: 1 usd: 0 Oct 25 13:41:07 TSPTONLDB01 kernel: [14317509.996487] Node 0 DMA32 per-cpu: Oct 25 13:41:07 TSPTONLDB01 kernel: [14317509.996488] CPU 0: hi: 186, btch: 31 usd: 0 Oct 25 13:41:07 TSPTONLDB01 kernel: [14317509.996489] CPU 1: hi: 186, btch: 31 usd: 5 Oct 25 13:41:07 TSPTONLDB01 kernel: [14317509.996490] Node 0 Normal per-cpu: Oct 25 13:41:07 TSPTONLDB01 kernel: [14317509.996491] CPU 0: hi: 186, btch: 31 usd: 30 Oct 25 13:41:07 TSPTONLDB01 kernel: [14317509.996492] CPU 1: hi: 186, btch: 31 usd: 144 Oct 25 13:41:07 TSPTONLDB01 kernel: [14317509.996495] active_anon:658762 inactive_anon:260562 isolated_anon:0 Oct 25 13:41:07 TSPTONLDB01 kernel: [14317509.996496] active_file:353 inactive_file:1716 isolated_file:0 Oct 25 13:41:07 TSPTONLDB01 kernel: [14317509.996496] unevictable:0 dirty:6 writeback:5 unstable:0 Oct 25 13:41:07 TSPTONLDB01 kernel: [14317509.996497] free:21800 slab_reclaimable:2214 slab_unreclaimable:10742 Oct 25 13:41:07 TSPTONLDB01 kernel: [14317509.996498] mapped:765 shmem:456 pagetables:6664 bounce:0 Oct 25 13:41:07 TSPTONLDB01 kernel: [14317509.996499] Node 0 DMA free:15880kB min:256kB low:320kB high:384kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15688kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes Oct 25 13:41:07 TSPTONLDB01 kernel: [14317509.996505] lowmem_reserve[]: 0 3000 4010 4010 Oct 25 13:41:07 TSPTONLDB01 kernel: [14317509.996507] Node 0 DMA32 free:54356kB min:50368kB low:62960kB high:75552kB active_anon:2275764kB inactive_anon:569032kB active_file:12kB inactive_file:88kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3072160kB mlocked:0kB dirty:0kB writeback:0kB mapped:32kB shmem:12kB slab_reclaimable:1896kB slab_unreclaimable:7060kB kernel_stack:2904kB pagetables:6588kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:611 all_unreclaimable? yes Oct 25 13:41:07 TSPTONLDB01 kernel: [14317509.996513] lowmem_reserve[]: 0 0 1010 1010 Oct 25 13:41:07 TSPTONLDB01 kernel: [14317509.996515] Node 0 Normal free:16964kB min:16956kB low:21192kB high:25432kB active_anon:359284kB inactive_anon:473216kB active_file:1400kB inactive_file:6776kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:1034240kB mlocked:0kB dirty:24kB writeback:20kB mapped:3028kB shmem:1812kB slab_reclaimable:6960kB slab_unreclaimable:35908kB kernel_stack:3624kB pagetables:20068kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:12878 all_unreclaimable? yes Oct 25 13:41:07 TSPTONLDB01 kernel: [14317509.996520] lowmem_reserve[]: 0 0 0 0 Oct 25 13:41:07 TSPTONLDB01 kernel: [14317509.996522] Node 0 DMA: 0*4kB 1*8kB 0*16kB 0*32kB 2*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15880kB Oct 25 13:41:07 TSPTONLDB01 kernel: [14317509.996544] Node 0 DMA32: 1308*4kB 973*8kB 448*16kB 181*32kB 91*64kB 43*128kB 11*256kB 8*512kB 8*1024kB 1*2048kB 0*4096kB = 54456kB Oct 25 13:41:07 TSPTONLDB01 kernel: [14317509.996549] Node 0 Normal: 439*4kB 303*8kB 169*16kB 83*32kB 18*64kB 5*128kB 4*256kB 3*512kB 3*1024kB 0*2048kB 0*4096kB = 16964kB Oct 25 13:41:07 TSPTONLDB01 kernel: [14317509.996553] 38343 total pagecache pages Oct 25 13:41:07 TSPTONLDB01 kernel: [14317509.996554] 35726 pages in swap cache Oct 25 13:41:07 TSPTONLDB01 kernel: [14317509.996556] Swap cache stats: add 5506519, delete 5470793, find 22920240/23085658 Oct 25 13:41:07 TSPTONLDB01 kernel: [14317509.996557] Free swap = 0kB Oct 25 13:41:07 TSPTONLDB01 kernel: [14317509.996557] Total swap = 2097148kB Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.006955] 1048560 pages RAM Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.006957] 67010 pages reserved Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.006958] 1978 pages shared Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.006959] 953768 pages non-shared Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.006960] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.006974] [ 571] 0 571 4223 1 1 -17 -1000 udevd Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.006981] [ 1689] 0 1689 998 1 0 0 0 acpid Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.006983] [ 1704] 100 1704 5444 117 0 0 0 dbus-daemon Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.006986] [ 1722] 0 1722 4824 62 0 0 0 syslog-ng Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.006988] [ 1725] 0 1725 1065 0 0 0 0 klogd Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.006991] [10038] 0 10038 31835 168 1 0 0 vmtoolsd Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.006994] [10096] 0 10096 15193 1 0 0 0 VGAuthService Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.006996] [10185] 101 10185 8608 146 0 0 0 hald Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.006999] [10188] 0 10188 27278 0 0 0 0 console-kit-dae Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007001] [10189] 0 10189 4505 2 0 0 0 hald-runner Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007004] [10284] 0 10284 8690 0 0 0 0 hald-addon-inpu Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007007] [10308] 0 10308 8690 17 0 0 0 hald-addon-stor Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007009] [10312] 101 10312 9129 0 1 0 0 hald-addon-acpi Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007012] [10313] 0 10313 8690 16 0 0 0 hald-addon-stor Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007014] [12046] 0 12046 7409 8 0 -17 -1000 auditd Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007017] [12048] 0 12048 3606 0 0 0 0 audispd Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007020] [12081] 0 12081 4723 16 0 0 0 rpcbind Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007022] [12083] 0 12083 2088 17 0 0 0 haveged Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007025] [12140] 0 12140 4216 1 1 -17 -1000 udevd Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007027] [12141] 0 12141 4216 1 0 -17 -1000 udevd Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007029] [12224] 0 12224 6759 1 1 0 0 mcelog Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007031] [12701] 0 12701 17296 28 0 -17 -1000 sshd Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007034] [12715] 0 12715 17933 0 1 0 0 cupsd Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007036] [12730] 0 12730 59557 71 0 0 0 nscd Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007038] [12798] 0 12798 9552 28 0 0 0 master Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007041] [12825] 51 12825 10052 31 1 0 0 qmgr Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007043] [12864] 0 12864 16711 47 1 0 0 gdm Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007046] [12878] 0 12878 27377 121 0 0 0 X Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007048] [12885] 0 12885 3697 15 0 0 0 cron Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007051] [12924] 0 12924 1121 2 1 0 0 mingetty Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007055] [12925] 0 12925 1121 2 1 0 0 mingetty Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007058] [12926] 0 12926 1121 2 0 0 0 mingetty Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007061] [12927] 0 12927 1121 2 1 0 0 mingetty Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007064] [12928] 0 12928 1121 2 1 0 0 mingetty Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007067] [12929] 0 12929 1121 2 0 0 0 mingetty Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007071] [13064] 107 13064 5548 0 1 0 0 dbus-launch Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007074] [13065] 107 13065 3362 0 1 0 0 dbus-daemon Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007077] [13066] 107 13066 39551 82 0 0 0 gnome-session Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007080] [13079] 107 13079 9863 67 0 0 0 gconfd-2 Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007083] [13083] 107 13083 31861 0 1 0 0 at-spi-registry Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007086] [13084] 107 13084 70831 2912 0 0 0 gnome-settings- Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007090] [13086] 107 13086 20207 0 1 0 0 bonobo-activati Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007092] [13094] 107 13094 57536 135 0 0 0 gnome-power-man Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007095] [13096] 107 13096 10854 0 0 0 0 gvfsd Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007098] [13097] 107 13097 54246 0 0 0 0 metacity Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007101] [21349] 1018 21349 2857 2 1 0 0 mysqld_safe Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007105] [12777] 1016 12777 2857 2 0 0 0 mysqld_safe Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007109] [18450] 0 18450 13384 49 1 0 0 gdm-simple-slav Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007112] [18452] 0 18452 66809 13462 1 0 0 X Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007115] [18489] 107 18489 5548 0 1 0 0 dbus-launch Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007118] [18518] 0 18518 24613 0 1 0 0 gdm-session-wor Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007121] [18540] 0 18540 39584 89 1 0 0 gnome-session Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007124] [18613] 0 18613 4178 19 0 0 0 gpg-agent Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007126] [18619] 0 18619 5548 0 1 0 0 dbus-launch Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007129] [18620] 0 18620 3429 0 0 0 0 dbus-daemon Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007132] [18623] 0 18623 9887 123 1 0 0 gconfd-2 Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007135] [18627] 0 18627 91696 332 0 0 0 gnome-settings- Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007138] [18631] 0 18631 23814 1 1 0 0 gnome-keyring-d Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007141] [18635] 0 18635 10908 0 1 0 0 gvfsd Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007144] [18639] 0 18639 16822 19 0 0 0 gvfs-fuse-daemo Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007147] [18645] 0 18645 51782 400 1 0 0 metacity Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007149] [18650] 0 18650 93659 621 1 0 0 gnome-panel Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007152] [18651] 0 18651 135905 13563 1 0 0 nautilus Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007155] [18653] 0 18653 36624 0 0 0 0 bonobo-activati Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007160] [18659] 0 18659 82105 455 1 0 0 main-menu Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007171] [18662] 0 18662 12083 0 1 0 0 gvfsd-trash Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007174] [18666] 0 18666 13609 1 0 0 0 gvfs-hal-volume Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007178] [18668] 0 18668 15774 1 1 0 0 gvfs-gphoto2-vo Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007181] [18677] 0 18677 75456 1 1 0 0 gnome-volume-co Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007184] [18681] 0 18681 56947 1 0 0 0 python Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007187] [18686] 0 18686 39298 113 0 0 0 vmtoolsd Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007189] [18688] 0 18688 57564 125 0 0 0 gnome-power-man Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007192] [18692] 0 18692 38020 114 1 0 0 gnome-screensav Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007195] [18693] 0 18693 48186 0 1 0 0 pulseaudio Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007198] [18718] 0 18718 10962 0 0 0 0 gvfsd-burn Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007201] [21417] 0 21417 59168 142 1 0 0 application-bro Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007204] [21420] 0 21420 57129 199 0 0 0 baobab Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007208] [21463] 0 21463 35883 1 0 0 0 gnomesu Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007211] [21464] 0 21464 13445 4 0 0 0 gnomesu-pam-bac Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007214] [21471] 0 21471 3221 1 0 0 0 yast2 Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007217] [21501] 0 21501 118060 114 0 0 0 y2base Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007221] [21621] 0 21621 55974 123 0 0 0 gnome-terminal Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007224] [21622] 0 21622 1616 0 0 0 0 gnome-pty-helpe Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007227] [21623] 0 21623 3505 1 0 0 0 bash Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007230] [21649] 0 21649 9244 1 0 0 0 gvfsd-metadata Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007233] [31142] 1016 31142 1100222 219744 1 0 0 mysqld Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007236] [31181] 1018 31181 1039509 163992 1 0 0 mysqld Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007239] [ 699] 1017 699 2856 98 0 0 0 mysqld_safe Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007242] [ 1625] 1017 1625 1076677 414298 1 0 0 mysqld Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007246] [ 1861] 1019 1861 641372 48223 1 0 0 java Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007284] [14971] 51 14971 10014 128 0 0 0 pickup Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007287] Out of memory: Kill process 31142 (mysqld) score 283 or sacrifice child Oct 25 13:41:07 TSPTONLDB01 kernel: [14317510.007358] Killed process 31142 (mysqld) total-vm:4400888kB, anon-rss:878976kB, file-rss:0kB Oct 25 13:45:45 TSPTONLDB01 kernel: [14317788.351765] vmtoolsd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 Oct 25 13:45:45 TSPTONLDB01 kernel: [14317788.351772] vmtoolsd cpuset=/ mems_allowed=0 Oct 25 13:45:45 TSPTONLDB01 kernel: [14317788.351777] Pid: 10038, comm: vmtoolsd Tainted: G X 3.0.13-0.27-default #1 Oct 25 13:45:45 TSPTONLDB01 kernel: [14317788.351780] Call Trace: Oct 25 13:45:45 TSPTONLDB01 kernel: [14317788.351803] [<ffffffff810048b5>] dump_trace+0x75/0x300 Oct 25 13:45:45 TSPTONLDB01 kernel: [14317788.351814] [<ffffffff8143ea0f>] dump_stack+0x69/0x6f Oct 25 13:45:45 TSPTONLDB01 kernel: [14317788.351825] [<ffffffff810f2b6f>] dump_header+0x8f/0x1f0 Oct 25 13:45:45 TSPTONLDB01 kernel: [14317788.351834] [<ffffffff810f3235>] oom_kill_process+0x95/0x180 Oct 25 13:45:45 TSPTONLDB01 kernel: [14317788.351841] [<ffffffff810f346c>] out_of_memory+0x14c/0x230 Oct 25 13:45:45 TSPTONLDB01 kernel: [14317788.351848] [<ffffffff810f8efb>] __alloc_pages_slowpath+0x7ab/0x7f0 Oct 25 13:45:45 TSPTONLDB01 kernel: [14317788.351854] [<ffffffff810f90f1>] __alloc_pages_nodemask+0x1b1/0x1c0 Oct 25 13:45:45 TSPTONLDB01 kernel: [14317788.351863] [<ffffffff811312b5>] alloc_pages_current+0xa5/0x120 Oct 25 13:45:45 TSPTONLDB01 kernel: [14317788.351871] [<ffffffff810f08f2>] filemap_fault+0x172/0x3d0 Oct 25 13:45:45 TSPTONLDB01 kernel: [14317788.351878] [<ffffffff8111916a>] __do_fault+0x7a/0x5d0 Oct 25 13:45:45 TSPTONLDB01 kernel: [14317788.351884] [<ffffffff8111975d>] handle_pte_fault+0x9d/0x230 Oct 25 13:45:45 TSPTONLDB01 kernel: [14317788.351893] [<ffffffff81445327>] do_page_fault+0x207/0x4c0 Oct 25 13:45:45 TSPTONLDB01 kernel: [14317788.351900] [<ffffffff81442065>] page_fault+0x25/0x30 Oct 25 13:45:45 TSPTONLDB01 kernel: [14317788.353040] DWARF2 unwinder stuck at page_fault+0x25/0x30 Oct 25 13:45:45 TSPTONLDB01 kernel: [14317788.353042] 通过日志发现发现在10月24日到11月份出现多次内存不足，操作系统 KILL 掉MYSQLD进程的情况。 在该服务器上有3个MYSQL实例，每个MYSQL实例的 INNODB 缓冲池设置为2GB，但操作系统的物理内存 只有4GB，另外在服务器上还有一个数据同步的 JAVA进程。结合日志分析，MYSQL 内存分配不合理，建议把服务器内存扩展到10GB。 建议MYSQL INNODB池计算方法： INNODB 缓冲池内存= 操作系统物理内存 - （预留给操作系统的内存1-2GB） - （JAVA进程的内存）-（MYSQL最大连接数*2MB） （编辑：核心网） 【声明】本站内容均来自网络，其相关言论仅代表作者个人观点，不代表本站立场。若无意侵犯到您的权利，请及时与联系站长删除相关内容! |
