pika 运行1个月左右会把 CentOS内存吃光，然后被操作系统kill

[yanjie108]

有没有和redis一样的配置设置最大占用内存？
系统配置8核16G
buffer用的默认配置

Pika write-buffer-size

write-buffer-size : 268435456

[KernelMaker]

你用的是哪个版本，pika 2.2.3修复了一个内存的问题

[yanjie108]

被kill 的版本是 2.2.0 那我升级一下
我们也有用2.2.3的应用，线上己运行8天，目前内存已经占用89%，pika2.2.3 是不是已经控制了最大�内存防止被操作系统kill掉呢
谢谢

[baotiao]

@yanjie108 确实是旧版本内存问题, 升级就可以了

[yanjie108]

@baotiao pika 2.2.3 运行了18天被操作系统kill了，生产系统有什么配置修改建议么，谢谢

[KernelMaker]

客户端连接数有多少呢？都是什么请求？

[yanjie108]

连接数快被kill的时候也没超过2000
只用到 kv(有过期时间)和hash 2种数据类型
查询主要用 MGET HMGET key的数量都有控制
qps高峰6000左右
下面是info信息(2017-07-22 17:00重启后的信息)

Server

pika_version:2.2.3
os:Linux 2.6.32-504.30.3.el6.x86_64 x86_64
arch_bits:64
process_id:31198
tcp_port:9221
thread_num:7
sync_thread_num:6
uptime_in_seconds:104172
uptime_in_days:2
config_file:conf/pika.conf

Data

db_size:162598712782
db_size_human:155066M
compression:snappy
used_memory:6234660920
used_memory_human:5945M
db_memtable_usage:229407720
db_tablereader_usage:6005253200

Log

log_size:5266483112
log_size_human:5022M
safety_purge:write2file42789
expire_logs_days:7
expire_logs_nums:10
binlog_offset:42799 20354602

Clients

connected_clients:916

Stats

total_connections_received:186159
instantaneous_ops_per_sec:2993
total_commands_processed:255012298
is_bgsaving:No, , 0
is_slots_reloading:No, , 0
is_scaning_keyspace:No
is_compact:No
compact_cron:

Replication(MASTER)

role:master
connected_slaves:0

Keyspace

Time:2017-07-23 21:52:57

kv keys:9683606
hash keys:14096
list keys:0
zset keys:0
set keys:0

[KernelMaker]

pika一个连接使用4M的内存，而且你用的是mget这样的请求，也有可能把writebuffer撑到大于4M，4M*2000就差不多8G多了，rocksdb又占了5G多，这就已经很大了，你把连接数降一下吧，内存不够

[yanjie108]

好的，thx

[yanjie108]

@KernelMaker 抽空帮我看一下，顺便给个优化建议 以下是运行6天的信息，内存也快耗光了
连接数已经在不影响业务的情况下控制最小了 有6台web服务一台最大300个连接
我看info Log显示5个G是占内存5个G么
万分感谢

-------------top-------------------
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
31198 root 20 0 20.3g 14g 4112 S 20.6 92.6 1626:07 pika
-------------free-------------------
free
total used free shared buffers cached
Mem: 16331772 16149600 182172 0 3132 671072
-/+ buffers/cache: 15475396 856376

-------------info-------------------

Server

pika_version:2.2.3
os:Linux 2.6.32-504.30.3.el6.x86_64 x86_64
arch_bits:64
process_id:31198
tcp_port:9221
thread_num:7
sync_thread_num:6
uptime_in_seconds:408230
uptime_in_days:6
config_file:conf/pika.conf

Data

db_size:162505129496
db_size_human:154976M
compression:snappy
used_memory:6197895568
used_memory_human:5910M
db_memtable_usage:204396456
db_tablereader_usage:5993499112

Log

log_size:5372015880
log_size_human:5123M
safety_purge:write2file43808
expire_logs_days:7
expire_logs_nums:10
binlog_offset:43818 36818515

Clients

connected_clients:1065

Stats

total_connections_received:721103
instantaneous_ops_per_sec:1771
total_commands_processed:1000207220
is_bgsaving:No, , 0
is_slots_reloading:No, , 0
is_scaning_keyspace:No
is_compact:No
compact_cron:

Replication(MASTER)

role:master
connected_slaves:0

Keyspace

Time:2017-07-27 10:20:38

kv keys:10512249
hash keys:14163
list keys:0
zset keys:0
set keys:0

[yanjie108]

@baotiao

[KernelMaker]

连接数降到300了，，内存呢占用多少呢？我看你发的，连接数1065，算上引擎占用的5g，一共使用14g内存差不多的

info里显示的内存占用仅仅是引擎的占用

[yanjie108]

@KernelMaker 抱歉我没有描述清楚
我的意思是一台web服务器链接池配置了最大300
1000左右的连接有办法防止被kill吗

谢谢

[yanjie108]

@KernelMaker 你们在生产环境没有碰到被kill的情况么
你们的场景是游戏 qps也不会低吧
今天又被kill了，已经是第三次了，服务只能坚持一星期。

[baotiao]

@yanjie108 能否把运行时刻的内存信息等等发给我们

[KernelMaker]

@yanjie108 如果连接数降不下去，那你只能调小max-open-files配置项，牺牲性能换取一些内存吧，默认5000，你可以调到1000试试

[yanjie108]

@baotiao @KernelMaker 问个小白问题，内存信息指什么？贴了2个日志一个现在的top 一个是被kill时，和pika相关的系统日志，大神有空帮看看
------------这个是现在top看的信息---------
top - 10:36:02 up 112 days, 0 min, 2 users, load average: 0.05, 0.02, 0.00
Tasks: 183 total, 1 running, 182 sleeping, 0 stopped, 0 zombie
Cpu(s): 2.0%us, 0.9%sy, 0.0%ni, 97.0%id, 0.0%wa, 0.0%hi, 0.1%si, 0.0%st
Mem: 16331772k total, 16178572k used, 153200k free, 28136k buffers
Swap: 0k total, 0k used, 0k free, 6990872k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1079 root 20 0 18.3g 8.1g 4976 S 18.3 52.2 232:26.26 pika

-----------下面是被kill相关的pika系统日志------------
Jul 30 19:52:13 localhost kernel: [9623794.839519] pika invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 30 19:52:13 localhost kernel: [9623794.839523] pika cpuset=/ mems_allowed=0
Jul 30 19:52:13 localhost kernel: [9623794.839525] Pid: 31213, comm: pika Tainted: G W --------------- 2.6.32-504.30.3.el6.x86_64 #1
Jul 30 19:52:13 localhost kernel: [9623794.839526] Call Trace:
Jul 30 19:52:13 localhost kernel: [9623794.839533] [] ? cpuset_print_task_mems_allowed+0x91/0xb0
Jul 30 19:52:13 localhost kernel: [9623794.839537] [] ? dump_header+0x90/0x1b0
Jul 30 19:52:13 localhost kernel: [9623794.839541] [] ? security_real_capable_noaudit+0x3c/0x70
Jul 30 19:52:13 localhost kernel: [9623794.839543] [] ? oom_kill_process+0x82/0x2a0
Jul 30 19:52:13 localhost kernel: [9623794.839545] [] ? select_bad_process+0xe1/0x120
Jul 30 19:52:13 localhost kernel: [9623794.839547] [] ? out_of_memory+0x220/0x3c0
Jul 30 19:52:13 localhost kernel: [9623794.839550] [] ? __alloc_pages_nodemask+0x89f/0x8d0
Jul 30 19:52:13 localhost kernel: [9623794.839554] [] ? alloc_pages_current+0xaa/0x110
Jul 30 19:52:13 localhost kernel: [9623794.839556] [] ? __page_cache_alloc+0x87/0x90
Jul 30 19:52:13 localhost kernel: [9623794.839558] [] ? find_get_page+0x1e/0xa0
Jul 30 19:52:13 localhost kernel: [9623794.839568] [] ? generic_file_aio_read+0x585/0x700
Jul 30 19:52:13 localhost kernel: [9623794.839572] [] ? do_sync_read+0xfa/0x140
Jul 30 19:52:13 localhost kernel: [9623794.839583] [] ? perf_event_task_sched_out+0x33/0x70
Jul 30 19:52:13 localhost kernel: [9623794.839587] [] ? autoremove_wake_function+0x0/0x40
Jul 30 19:52:13 localhost kernel: [9623794.839593] [] ? security_file_permission+0x16/0x20
Jul 30 19:52:13 localhost kernel: [9623794.839595] [] ? vfs_read+0xb5/0x1a0
Jul 30 19:52:13 localhost kernel: [9623794.839597] [] ? sys_pread64+0x82/0xa0
Jul 30 19:52:13 localhost kernel: [9623794.839602] [] ? system_call_fastpath+0x16/0x1b
Jul 30 19:52:13 localhost kernel: [9623794.839603] Mem-Info:
Jul 30 19:52:13 localhost kernel: [9623794.839604] Node 0 DMA per-cpu:
Jul 30 19:52:13 localhost kernel: [9623794.839606] CPU 0: hi: 0, btch: 1 usd: 0
Jul 30 19:52:13 localhost kernel: [9623794.839607] CPU 1: hi: 0, btch: 1 usd: 0
Jul 30 19:52:13 localhost kernel: [9623794.839608] CPU 2: hi: 0, btch: 1 usd: 0
Jul 30 19:52:13 localhost kernel: [9623794.839609] CPU 3: hi: 0, btch: 1 usd: 0
Jul 30 19:52:13 localhost kernel: [9623794.839610] CPU 4: hi: 0, btch: 1 usd: 0
Jul 30 19:52:13 localhost kernel: [9623794.839611] CPU 5: hi: 0, btch: 1 usd: 0
Jul 30 19:52:13 localhost kernel: [9623794.839612] CPU 6: hi: 0, btch: 1 usd: 0
Jul 30 19:52:13 localhost kernel: [9623794.839613] CPU 7: hi: 0, btch: 1 usd: 0
Jul 30 19:52:13 localhost kernel: [9623794.839614] Node 0 DMA32 per-cpu:
Jul 30 19:52:13 localhost kernel: [9623794.839616] CPU 0: hi: 186, btch: 31 usd: 60
Jul 30 19:52:13 localhost kernel: [9623794.839617] CPU 1: hi: 186, btch: 31 usd: 62
Jul 30 19:52:13 localhost kernel: [9623794.839618] CPU 2: hi: 186, btch: 31 usd: 47
Jul 30 19:52:13 localhost kernel: [9623794.839619] CPU 3: hi: 186, btch: 31 usd: 44
Jul 30 19:52:13 localhost kernel: [9623794.839620] CPU 4: hi: 186, btch: 31 usd: 94
Jul 30 19:52:13 localhost kernel: [9623794.839621] CPU 5: hi: 186, btch: 31 usd: 30
Jul 30 19:52:13 localhost kernel: [9623794.839622] CPU 6: hi: 186, btch: 31 usd: 22
Jul 30 19:52:13 localhost kernel: [9623794.839623] CPU 7: hi: 186, btch: 31 usd: 64
Jul 30 19:52:13 localhost kernel: [9623794.839624] Node 0 Normal per-cpu:
Jul 30 19:52:13 localhost kernel: [9623794.839626] CPU 0: hi: 186, btch: 31 usd: 176
Jul 30 19:52:13 localhost kernel: [9623794.839627] CPU 1: hi: 186, btch: 31 usd: 10
Jul 30 19:52:13 localhost kernel: [9623794.839628] CPU 2: hi: 186, btch: 31 usd: 41
Jul 30 19:52:13 localhost kernel: [9623794.839629] CPU 3: hi: 186, btch: 31 usd: 52
Jul 30 19:52:13 localhost kernel: [9623794.839630] CPU 4: hi: 186, btch: 31 usd: 50
Jul 30 19:52:13 localhost kernel: [9623794.839631] CPU 5: hi: 186, btch: 31 usd: 69
Jul 30 19:52:13 localhost kernel: [9623794.839632] CPU 6: hi: 186, btch: 31 usd: 136
Jul 30 19:52:13 localhost kernel: [9623794.839633] CPU 7: hi: 186, btch: 31 usd: 71
Jul 30 19:52:13 localhost kernel: [9623794.839635] active_anon:3986522 inactive_anon:171 isolated_anon:0
Jul 30 19:52:13 localhost kernel: [9623794.839636] active_file:285 inactive_file:1788 isolated_file:0
Jul 30 19:52:13 localhost kernel: [9623794.839636] unevictable:0 dirty:170 writeback:0 unstable:0
Jul 30 19:52:13 localhost kernel: [9623794.839637] free:33382 slab_reclaimable:4700 slab_unreclaimable:7869
Jul 30 19:52:13 localhost kernel: [9623794.839637] mapped:312 shmem:172 pagetables:10759 bounce:0
Jul 30 19:52:13 localhost kernel: [9623794.839639] Node 0 DMA free:15692kB min:60kB low:72kB high:88kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15304kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 30 19:52:13 localhost kernel: [9623794.839644] lowmem_reserve[]: 0 3512 16137 16137
Jul 30 19:52:13 localhost kernel: [9623794.839646] Node 0 DMA32 free:65108kB min:14692kB low:18364kB high:22036kB active_anon:3117388kB inactive_anon:0kB active_file:788kB inactive_file:6920kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3596500kB mlocked:0kB dirty:352kB writeback:0kB mapped:524kB shmem:0kB slab_reclaimable:3104kB slab_unreclaimable:2688kB kernel_stack:128kB pagetables:2292kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1207 all_unreclaimable? yes
Jul 30 19:52:13 localhost kernel: [9623794.839652] lowmem_reserve[]: 0 0 12625 12625
Jul 30 19:52:13 localhost kernel: [9623794.839654] Node 0 Normal free:52728kB min:52824kB low:66028kB high:79236kB active_anon:12828700kB inactive_anon:684kB active_file:352kB inactive_file:232kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:12928000kB mlocked:0kB dirty:328kB writeback:0kB mapped:724kB shmem:688kB slab_reclaimable:15696kB slab_unreclaimable:28788kB kernel_stack:1544kB pagetables:40744kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:896 all_unreclaimable? yes
Jul 30 19:52:13 localhost kernel: [9623794.839659] lowmem_reserve[]: 0 0 0 0
Jul 30 19:52:13 localhost kernel: [9623794.839661] Node 0 DMA: 34kB 28kB 116kB 132kB 264kB 1128kB 0256kB 0512kB 11024kB 12048kB 34096kB = 15692kB
Jul 30 19:52:13 localhost kernel: [9623794.839666] Node 0 DMA32: 3874kB 5188kB 34916kB 36432kB 22864kB 88128kB 25256kB 4512kB 01024kB 12048kB 14096kB = 63372kB
Jul 30 19:52:13 localhost kernel: [9623794.839671] Node 0 Normal: 121164kB 238kB 116kB 132kB 164kB 1128kB 1256kB 1512kB 11024kB 12048kB 04096kB = 52728kB
Jul 30 19:52:13 localhost kernel: [9623794.839676] 2204 total pagecache pages
Jul 30 19:52:13 localhost kernel: [9623794.839677] 0 pages in swap cache
Jul 30 19:52:13 localhost kernel: [9623794.839678] Swap cache stats: add 0, delete 0, find 0/0
Jul 30 19:52:13 localhost kernel: [9623794.839679] Free swap = 0kB
Jul 30 19:52:13 localhost kernel: [9623794.839680] Total swap = 0kB
Jul 30 19:52:13 localhost kernel: [9623794.863171] 4194303 pages RAM
Jul 30 19:52:13 localhost kernel: [9623794.863173] 111360 pages reserved
Jul 30 19:52:13 localhost kernel: [9623794.863174] 1904 pages shared
Jul 30 19:52:13 localhost kernel: [9623794.863175] 4044023 pages non-shared
Jul 30 19:52:13 localhost kernel: [9623794.863176] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 30 19:52:13 localhost kernel: [9623794.863190] [ 438] 0 438 2670 112 6 -17 -1000 udevd
Jul 30 19:52:13 localhost kernel: [9623794.863192] [ 748] 0 748 1019 22 6 0 0 agetty
Jul 30 19:52:13 localhost kernel: [9623794.863195] [ 1038] 81 1038 5351 58 1 0 0 dbus-daemon
Jul 30 19:52:13 localhost kernel: [9623794.863196] [ 1067] 0 1067 1020 28 0 0 0 acpid
Jul 30 19:52:13 localhost kernel: [9623794.863198] [ 1178] 0 1178 20323 226 6 0 0 master
Jul 30 19:52:13 localhost kernel: [9623794.863200] [ 1191] 89 1191 20386 233 6 0 0 qmgr
Jul 30 19:52:13 localhost kernel: [9623794.863202] [ 1194] 0 1194 5124 163 6 0 0 crond
Jul 30 19:52:13 localhost kernel: [9623794.863204] [ 1207] 0 1207 5385 47 6 0 0 atd
Jul 30 19:52:13 localhost kernel: [9623794.863206] [ 1355] 0 1355 24666 111 3 0 0 sgagent
Jul 30 19:52:13 localhost kernel: [9623794.863208] [ 1593] 0 1593 1016 22 7 0 0 mingetty
Jul 30 19:52:13 localhost kernel: [9623794.863210] [ 1596] 0 1596 1016 22 7 0 0 mingetty
Jul 30 19:52:13 localhost kernel: [9623794.863211] [ 1598] 0 1598 1016 22 3 0 0 mingetty
Jul 30 19:52:13 localhost kernel: [9623794.863213] [ 1600] 0 1600 1016 22 3 0 0 mingetty
Jul 30 19:52:13 localhost kernel: [9623794.863215] [ 1604] 0 1604 2670 117 6 -17 -1000 udevd
Jul 30 19:52:13 localhost kernel: [9623794.863217] [ 1605] 0 1605 2669 110 0 -17 -1000 udevd
Jul 30 19:52:13 localhost kernel: [9623794.863219] [ 1606] 0 1606 1016 22 7 0 0 mingetty
Jul 30 19:52:13 localhost kernel: [9623794.863221] [ 1608] 0 1608 1016 21 3 0 0 mingetty
Jul 30 19:52:13 localhost kernel: [9623794.863223] [ 9286] 500 9286 4504 48 3 0 0 zabbix_agentd
Jul 30 19:52:13 localhost kernel: [9623794.863225] [ 9288] 500 9288 4504 191 6 0 0 zabbix_agentd
Jul 30 19:52:13 localhost kernel: [9623794.863227] [ 9289] 500 9289 4504 72 6 0 0 zabbix_agentd
Jul 30 19:52:13 localhost kernel: [9623794.863229] [ 9290] 500 9290 4504 72 6 0 0 zabbix_agentd
Jul 30 19:52:13 localhost kernel: [9623794.863230] [ 9291] 500 9291 4504 72 6 0 0 zabbix_agentd
Jul 30 19:52:13 localhost kernel: [9623794.863232] [ 9292] 500 9292 4507 62 6 0 0 zabbix_agentd
Jul 30 19:52:13 localhost kernel: [9623794.863235] [27750] 0 27750 23289 76 6 -17 -1000 auditd
Jul 30 19:52:13 localhost kernel: [9623794.863237] [19731] 0 19731 62992 1253 6 0 0 rsyslogd
Jul 30 19:52:13 localhost kernel: [9623794.863239] [24617] 0 24617 16665 178 6 -17 -1000 sshd
Jul 30 19:52:13 localhost kernel: [9623794.863241] [19232] 0 19232 9462 4671 6 0 0 secu-tcs-agent
Jul 30 19:52:13 localhost kernel: [9623794.863243] [19241] 0 19241 768 342 6 0 0 sap1004
Jul 30 19:52:13 localhost kernel: [9623794.863245] [19243] 0 19243 7866 3075 3 0 0 sap1005
Jul 30 19:52:13 localhost kernel: [9623794.863247] [19620] 0 19620 11827 1440 6 0 0 barad_agent
Jul 30 19:52:13 localhost kernel: [9623794.863249] [19628] 0 19628 15060 1839 6 0 0 barad_agent
Jul 30 19:52:13 localhost kernel: [9623794.863251] [19629] 0 19629 109425 2553 6 0 0 barad_agent
Jul 30 19:52:13 localhost kernel: [9623794.863253] [23964] 0 23964 1792 1408 6 0 0 sap1002
Jul 30 19:52:13 localhost kernel: [9623794.863255] [ 4812] 0 4812 9754 3737 6 0 0 sap1009
Jul 30 19:52:13 localhost kernel: [9623794.863257] [31198] 0 31198 5300488 3965305 0 0 0 pika
Jul 30 19:52:13 localhost kernel: [9623794.863259] [18891] 0 18891 2331 92 6 0 0 watchdog.sh
Jul 30 19:52:13 localhost kernel: [9623794.863261] [31004] 89 31004 20347 220 6 0 0 pickup
Jul 30 19:52:13 localhost kernel: [9623794.863263] [ 934] 0 934 1018 21 1 0 0 sleep
Jul 30 19:52:13 localhost kernel: [9623794.863265] Out of memory: Kill process 31198 (pika) score 943 or sacrifice child
Jul 30 19:52:13 localhost kernel: [9623794.863268] Killed process 31198, UID 0, (pika) total-vm:21201952kB, anon-rss:15860312kB, file-rss:908kB
Jul 30 19:52:13 localhost kernel: [9623794.865541] pika invoked oom-killer: gfp_mask=0xd0, order=0, oom_adj=0, oom_score_adj=0
Jul 30 19:52:13 localhost kernel: [9623794.865543] pika cpuset=/ mems_allowed=0
Jul 30 19:52:13 localhost kernel: [9623794.865546] Pid: 31222, comm: pika Tainted: G W --------------- 2.6.32-504.30.3.el6.x86_64 #1
Jul 30 19:52:13 localhost kernel: [9623794.865547] Call Trace:
Jul 30 19:52:13 localhost kernel: [9623794.865553] [] ? cpuset_print_task_mems_allowed+0x91/0xb0
Jul 30 19:52:13 localhost kernel: [9623794.865557] [] ? dump_header+0x90/0x1b0
Jul 30 19:52:13 localhost kernel: [9623794.865560] [] ? security_real_capable_noaudit+0x3c/0x70
Jul 30 19:52:13 localhost kernel: [9623794.865563] [] ? oom_kill_process+0x82/0x2a0
Jul 30 19:52:13 localhost kernel: [9623794.865564] [] ? select_bad_process+0xe1/0x120
Jul 30 19:52:13 localhost kernel: [9623794.865567] [] ? out_of_memory+0x220/0x3c0
Jul 30 19:52:13 localhost kernel: [9623794.865569] [] ? __alloc_pages_nodemask+0x89f/0x8d0
Jul 30 19:52:13 localhost kernel: [9623794.865574] [] ? kmem_getpages+0x62/0x170
Jul 30 19:52:13 localhost kernel: [9623794.865576] [] ? fallback_alloc+0x1ba/0x270
Jul 30 19:52:13 localhost kernel: [9623794.865578] [] ? cache_grow+0x2cf/0x320
Jul 30 19:52:13 localhost kernel: [9623794.865581] [] ? ____cache_alloc_node+0x99/0x160
Jul 30 19:52:13 localhost kernel: [9623794.865583] [] ? kmem_cache_alloc+0x123/0x190
Jul 30 19:52:13 localhost kernel: [9623794.865586] [] ? taskstats_exit+0x305/0x390
Jul 30 19:52:13 localhost kernel: [9623794.865590] [] ? do_exit+0x157/0x870
Jul 30 19:52:13 localhost kernel: [9623794.865593] [] ? do_group_exit+0x58/0xd0
Jul 30 19:52:13 localhost kernel: [9623794.865597] [] ? get_signal_to_deliver+0x1f6/0x460
Jul 30 19:52:13 localhost kernel: [9623794.865601] [] ? do_signal+0x75/0x870
Jul 30 19:52:13 localhost kernel: [9623794.865605] [] ? cp_new_stat+0xe4/0x100
Jul 30 19:52:13 localhost kernel: [9623794.865608] [] ? pvclock_clocksource_read+0x58/0xd0
Jul 30 19:52:13 localhost kernel: [9623794.865611] [] ? sys_futex+0x7b/0x170
Jul 30 19:52:13 localhost kernel: [9623794.865613] [] ? do_notify_resume+0x90/0xc0
Jul 30 19:52:13 localhost kernel: [9623794.865616] [] ? int_signal+0x12/0x17
Jul 30 19:52:13 localhost kernel: [9623794.865617] Mem-Info:
Jul 30 19:52:13 localhost kernel: [9623794.865618] Node 0 DMA per-cpu:
Jul 30 19:52:13 localhost kernel: [9623794.865620] CPU 0: hi: 0, btch: 1 usd: 0
Jul 30 19:52:13 localhost kernel: [9623794.865621] CPU 1: hi: 0, btch: 1 usd: 0
Jul 30 19:52:13 localhost kernel: [9623794.865622] CPU 2: hi: 0, btch: 1 usd: 0
Jul 30 19:52:13 localhost kernel: [9623794.865623] CPU 3: hi: 0, btch: 1 usd: 0
Jul 30 19:52:13 localhost kernel: [9623794.865624] CPU 4: hi: 0, btch: 1 usd: 0
Jul 30 19:52:13 localhost kernel: [9623794.865625] CPU 5: hi: 0, btch: 1 usd: 0
Jul 30 19:52:13 localhost kernel: [9623794.865626] CPU 6: hi: 0, btch: 1 usd: 0
Jul 30 19:52:13 localhost kernel: [9623794.865627] CPU 7: hi: 0, btch: 1 usd: 0
Jul 30 19:52:13 localhost kernel: [9623794.865628] Node 0 DMA32 per-cpu:
Jul 30 19:52:13 localhost kernel: [9623794.865630] CPU 0: hi: 186, btch: 31 usd: 60
Jul 30 19:52:13 localhost kernel: [9623794.865632] CPU 1: hi: 186, btch: 31 usd: 62
Jul 30 19:52:13 localhost kernel: [9623794.865633] CPU 2: hi: 186, btch: 31 usd: 47
Jul 30 19:52:13 localhost kernel: [9623794.865635] CPU 3: hi: 186, btch: 31 usd: 44
Jul 30 19:52:13 localhost kernel: [9623794.865637] CPU 4: hi: 186, btch: 31 usd: 94
Jul 30 19:52:13 localhost kernel: [9623794.865638] CPU 5: hi: 186, btch: 31 usd: 30
Jul 30 19:52:13 localhost kernel: [9623794.865639] CPU 6: hi: 186, btch: 31 usd: 22
Jul 30 19:52:13 localhost kernel: [9623794.865640] CPU 7: hi: 186, btch: 31 usd: 64
Jul 30 19:52:13 localhost kernel: [9623794.865641] Node 0 Normal per-cpu:
Jul 30 19:52:13 localhost kernel: [9623794.865642] CPU 0: hi: 186, btch: 31 usd: 175
Jul 30 19:52:13 localhost kernel: [9623794.865643] CPU 1: hi: 186, btch: 31 usd: 10
Jul 30 19:52:13 localhost kernel: [9623794.865644] CPU 2: hi: 186, btch: 31 usd: 41
Jul 30 19:52:13 localhost kernel: [9623794.865645] CPU 3: hi: 186, btch: 31 usd: 52
Jul 30 19:52:13 localhost kernel: [9623794.865647] CPU 4: hi: 186, btch: 31 usd: 50
Jul 30 19:52:13 localhost kernel: [9623794.865648] CPU 5: hi: 186, btch: 31 usd: 69
Jul 30 19:52:13 localhost kernel: [9623794.865649] CPU 6: hi: 186, btch: 31 usd: 46
Jul 30 19:52:13 localhost kernel: [9623794.865650] CPU 7: hi: 186, btch: 31 usd: 71
Jul 30 19:52:13 localhost kernel: [9623794.865652] active_anon:3986522 inactive_anon:171 isolated_anon:0
Jul 30 19:52:13 localhost kernel: [9623794.865653] active_file:285 inactive_file:1788 isolated_file:0
Jul 30 19:52:13 localhost kernel: [9623794.865654] unevictable:0 dirty:170 writeback:0 unstable:0
Jul 30 19:52:13 localhost kernel: [9623794.865654] free:33382 slab_reclaimable:4700 slab_unreclaimable:7869
Jul 30 19:52:13 localhost kernel: [9623794.865655] mapped:312 shmem:172 pagetables:10759 bounce:0
Jul 30 19:52:13 localhost kernel: [9623794.865656] Node 0 DMA free:15692kB min:60kB low:72kB high:88kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15304kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 30 19:52:13 localhost kernel: [9623794.865661] lowmem_reserve[]: 0 3512 16137 16137
Jul 30 19:52:13 localhost kernel: [9623794.865663] Node 0 DMA32 free:65108kB min:14692kB low:18364kB high:22036kB active_anon:3117388kB inactive_anon:0kB active_file:788kB inactive_file:6920kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3596500kB mlocked:0kB dirty:352kB writeback:0kB mapped:524kB shmem:0kB slab_reclaimable:3104kB slab_unreclaimable:2688kB kernel_stack:128kB pagetables:2292kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1207 all_unreclaimable? yes
Jul 30 19:52:13 localhost kernel: [9623794.865668] lowmem_reserve[]: 0 0 12625 12625
Jul 30 19:52:13 localhost kernel: [9623794.865670] Node 0 Normal free:52728kB min:52824kB low:66028kB high:79236kB active_anon:12828700kB inactive_anon:684kB active_file:352kB inactive_file:232kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:12928000kB mlocked:0kB dirty:328kB writeback:0kB mapped:724kB shmem:688kB slab_reclaimable:15696kB slab_unreclaimable:28788kB kernel_stack:1544kB pagetables:40744kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:896 all_unreclaimable? yes
Jul 30 19:52:13 localhost kernel: [9623794.865675] lowmem_reserve[]: 0 0 0 0
Jul 30 19:52:13 localhost kernel: [9623794.865676] Node 0 DMA: 34kB 28kB 116kB 132kB 264kB 1128kB 0256kB 0512kB 11024kB 12048kB 34096kB = 15692kB
Jul 30 19:52:13 localhost kernel: [9623794.865681] Node 0 DMA32: 3874kB 5188kB 34916kB 36432kB 22864kB 88128kB 25256kB 4512kB 01024kB 12048kB 14096kB = 63372kB
Jul 30 19:52:13 localhost kernel: [9623794.865686] Node 0 Normal: 120854kB 238kB 116kB 132kB 164kB 1128kB 1256kB 1512kB 11024kB 12048kB 04096kB = 52604kB
Jul 30 19:52:13 localhost kernel: [9623794.865692] 2204 total pagecache pages
Jul 30 19:52:13 localhost kernel: [9623794.865693] 0 pages in swap cache
Jul 30 19:52:13 localhost kernel: [9623794.865694] Swap cache stats: add 0, delete 0, find 0/0
Jul 30 19:52:13 localhost kernel: [9623794.865695] Free swap = 0kB
Jul 30 19:52:13 localhost kernel: [9623794.865695] Total swap = 0kB
Jul 30 19:52:13 localhost kernel: [9623794.887692] 4194303 pages RAM
Jul 30 19:52:13 localhost kernel: [9623794.887694] 111360 pages reserved
Jul 30 19:52:13 localhost kernel: [9623794.887695] 1911 pages shared
Jul 30 19:52:13 localhost kernel: [9623794.887695] 4044139 pages non-shared
Jul 30 19:52:13 localhost kernel: [9623794.887696] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 30 19:52:13 localhost kernel: [9623794.887712] [ 438] 0 438 2670 112 6 -17 -1000 udevd
Jul 30 19:52:13 localhost kernel: [9623794.887714] [ 748] 0 748 1019 22 6 0 0 agetty
Jul 30 19:52:13 localhost kernel: [9623794.887716] [ 1038] 81 1038 5351 58 1 0 0 dbus-daemon
Jul 30 19:52:13 localhost kernel: [9623794.887718] [ 1067] 0 1067 1020 28 0 0 0 acpid
Jul 30 19:52:13 localhost kernel: [9623794.887720] [ 1178] 0 1178 20323 226 6 0 0 master
Jul 30 19:52:13 localhost kernel: [9623794.887722] [ 1191] 89 1191 20386 233 6 0 0 qmgr
Jul 30 19:52:13 localhost kernel: [9623794.887724] [ 1194] 0 1194 5124 163 6 0 0 crond
Jul 30 19:52:13 localhost kernel: [9623794.887725] [ 1207] 0 1207 5385 47 6 0 0 atd
Jul 30 19:52:13 localhost kernel: [9623794.887727] [ 1355] 0 1355 24666 111 3 0 0 sgagent
Jul 30 19:52:13 localhost kernel: [9623794.887729] [ 1593] 0 1593 1016 22 7 0 0 mingetty
Jul 30 19:52:13 localhost kernel: [9623794.887731] [ 1596] 0 1596 1016 22 7 0 0 mingetty
Jul 30 19:52:13 localhost kernel: [9623794.887733] [ 1598] 0 1598 1016 22 3 0 0 mingetty
Jul 30 19:52:13 localhost kernel: [9623794.887735] [ 1600] 0 1600 1016 22 3 0 0 mingetty
Jul 30 19:52:13 localhost kernel: [9623794.887737] [ 1604] 0 1604 2670 117 6 -17 -1000 udevd
Jul 30 19:52:13 localhost kernel: [9623794.887739] [ 1605] 0 1605 2669 110 0 -17 -1000 udevd
Jul 30 19:52:13 localhost kernel: [9623794.887740] [ 1606] 0 1606 1016 22 7 0 0 mingetty
Jul 30 19:52:13 localhost kernel: [9623794.887742] [ 1608] 0 1608 1016 21 3 0 0 mingetty
Jul 30 19:52:13 localhost kernel: [9623794.887744] [ 9286] 500 9286 4504 48 3 0 0 zabbix_agentd
Jul 30 19:52:13 localhost kernel: [9623794.887746] [ 9288] 500 9288 4504 191 6 0 0 zabbix_agentd
Jul 30 19:52:13 localhost kernel: [9623794.887748] [ 9289] 500 9289 4504 72 6 0 0 zabbix_agentd
Jul 30 19:52:13 localhost kernel: [9623794.887750] [ 9290] 500 9290 4504 72 6 0 0 zabbix_agentd
Jul 30 19:52:13 localhost kernel: [9623794.887752] [ 9291] 500 9291 4504 72 6 0 0 zabbix_agentd
Jul 30 19:52:13 localhost kernel: [9623794.887753] [ 9292] 500 9292 4507 62 6 0 0 zabbix_agentd
Jul 30 19:52:13 localhost kernel: [9623794.887756] [27750] 0 27750 23289 76 6 -17 -1000 auditd
Jul 30 19:52:13 localhost kernel: [9623794.887758] [19731] 0 19731 62992 1253 6 0 0 rsyslogd
Jul 30 19:52:13 localhost kernel: [9623794.887760] [24617] 0 24617 16665 178 6 -17 -1000 sshd
Jul 30 19:52:13 localhost kernel: [9623794.887762] [19232] 0 19232 9462 4671 6 0 0 secu-tcs-agent
Jul 30 19:52:13 localhost kernel: [9623794.887764] [19241] 0 19241 768 342 6 0 0 sap1004
Jul 30 19:52:13 localhost kernel: [9623794.887766] [19243] 0 19243 7866 3075 6 0 0 sap1005
Jul 30 19:52:13 localhost kernel: [9623794.887768] [19620] 0 19620 11827 1440 6 0 0 barad_agent
Jul 30 19:52:13 localhost kernel: [9623794.887770] [19628] 0 19628 15060 1839 6 0 0 barad_agent
Jul 30 19:52:13 localhost kernel: [9623794.887772] [19629] 0 19629 109425 2555 6 0 0 barad_agent
Jul 30 19:52:13 localhost kernel: [9623794.887773] [23964] 0 23964 1792 1408 6 0 0 sap1002
Jul 30 19:52:13 localhost kernel: [9623794.887775] [ 4812] 0 4812 9754 3738 6 0 0 sap1009
Jul 30 19:52:13 localhost kernel: [9623794.887777] [31214] 0 31198 5300488 3965305 6 0 0 pika
Jul 30 19:52:13 localhost kernel: [9623794.887779] [18891] 0 18891 2331 92 6 0 0 watchdog.sh
Jul 30 19:52:13 localhost kernel: [9623794.887781] [31004] 89 31004 20347 220 6 0 0 pickup
Jul 30 19:52:13 localhost kernel: [9623794.887783] [ 934] 0 934 1018 21 1 0 0 sleep
Jul 30 19:52:13 localhost kernel: [9623794.888367] barad_agent invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 30 19:52:13 localhost kernel: [9623794.888371] barad_agent cpuset=/ mems_allowed=0
Jul 30 19:52:13 localhost kernel: [9623794.888373] Pid: 19638, comm: barad_agent Tainted: G W --------------- 2.6.32-504.30.3.el6.x86_64 #1
Jul 30 19:52:13 localhost kernel: [9623794.888374] Call Trace:
Jul 30 19:52:13 localhost kernel: [9623794.888380] [] ? cpuset_print_task_mems_allowed+0x91/0xb0
Jul 30 19:52:13 localhost kernel: [9623794.888384] [] ? dump_header+0x90/0x1b0
Jul 30 19:52:13 localhost kernel: [9623794.888388] [] ? security_real_capable_noaudit+0x3c/0x70
Jul 30 19:52:13 localhost kernel: [9623794.888390] [] ? oom_kill_process+0x82/0x2a0
Jul 30 19:52:13 localhost kernel: [9623794.888392] [] ? select_bad_process+0xe1/0x120
Jul 30 19:52:13 localhost kernel: [9623794.888394] [] ? out_of_memory+0x220/0x3c0
Jul 30 19:52:13 localhost kernel: [9623794.888397] [] ? __alloc_pages_nodemask+0x89f/0x8d0
Jul 30 19:52:13 localhost kernel: [9623794.888401] [] ? alloc_pages_current+0xaa/0x110
Jul 30 19:52:13 localhost kernel: [9623794.888403] [] ? __page_cache_alloc+0x87/0x90
Jul 30 19:52:13 localhost kernel: [9623794.888405] [] ? find_get_page+0x1e/0xa0
Jul 30 19:52:13 localhost kernel: [9623794.888406] [] ? filemap_fault+0x1a7/0x500
Jul 30 19:52:13 localhost kernel: [9623794.888411] [] ? __do_fault+0x54/0x530
Jul 30 19:52:13 localhost kernel: [9623794.888413] [] ? handle_pte_fault+0xf7/0xb00
Jul 30 19:52:13 localhost kernel: [9623794.888418] [] ? mntput_no_expire+0x30/0x110
Jul 30 19:52:13 localhost kernel: [9623794.888420] [] ? handle_mm_fault+0x299/0x3d0
Jul 30 19:52:13 localhost kernel: [9623794.888424] [] ? __do_page_fault+0x146/0x500
Jul 30 19:52:13 localhost kernel: [9623794.888428] [] ? kvm_clock_read+0x1c/0x20
Jul 30 19:52:13 localhost kernel: [9623794.888431] [] ? kvm_clock_get_cycles+0x9/0x10
Jul 30 19:52:13 localhost kernel: [9623794.888435] [] ? ktime_get_ts+0xb1/0xf0
Jul 30 19:52:13 localhost kernel: [9623794.888438] [] ? pvclock_clocksource_read+0x58/0xd0
Jul 30 19:52:13 localhost kernel: [9623794.888440] [] ? kvm_clock_read+0x1c/0x20
Jul 30 19:52:13 localhost kernel: [9623794.888447] [] ? kvm_clock_get_cycles+0x9/0x10
Jul 30 19:52:13 localhost kernel: [9623794.888449] [] ? getnstimeofday+0x57/0xe0
Jul 30 19:52:13 localhost kernel: [9623794.888453] [] ? do_page_fault+0x3e/0xa0
Jul 30 19:52:13 localhost kernel: [9623794.888456] [] ? page_fault+0x25/0x30
Jul 30 19:52:13 localhost kernel: [9623794.888457] Mem-Info:
Jul 30 19:52:13 localhost kernel: [9623794.888458] Node 0 DMA per-cpu:
Jul 30 19:52:13 localhost kernel: [9623794.888459] CPU 0: hi: 0, btch: 1 usd: 0
Jul 30 19:52:13 localhost kernel: [9623794.888461] CPU 1: hi: 0, btch: 1 usd: 0
Jul 30 19:52:13 localhost kernel: [9623794.888462] CPU 2: hi: 0, btch: 1 usd: 0
Jul 30 19:52:13 localhost kernel: [9623794.888463] CPU 3: hi: 0, btch: 1 usd: 0
Jul 30 19:52:13 localhost kernel: [9623794.888464] CPU 4: hi: 0, btch: 1 usd: 0
Jul 30 19:52:13 localhost kernel: [9623794.888465] CPU 5: hi: 0, btch: 1 usd: 0
Jul 30 19:52:13 localhost kernel: [9623794.888466] CPU 6: hi: 0, btch: 1 usd: 0
Jul 30 19:52:13 localhost kernel: [9623794.888467] CPU 7: hi: 0, btch: 1 usd: 0
Jul 30 19:52:13 localhost kernel: [9623794.888468] Node 0 DMA32 per-cpu:
Jul 30 19:52:13 localhost kernel: [9623794.888469] CPU 0: hi: 186, btch: 31 usd: 60
Jul 30 19:52:13 localhost kernel: [9623794.888470] CPU 1: hi: 186, btch: 31 usd: 62
Jul 30 19:52:13 localhost kernel: [9623794.888471] CPU 2: hi: 186, btch: 31 usd: 47
Jul 30 19:52:13 localhost kernel: [9623794.888473] CPU 3: hi: 186, btch: 31 usd: 44
Jul 30 19:52:13 localhost kernel: [9623794.888474] CPU 4: hi: 186, btch: 31 usd: 94
Jul 30 19:52:13 localhost kernel: [9623794.888475] CPU 5: hi: 186, btch: 31 usd: 30
Jul 30 19:52:13 localhost kernel: [9623794.888476] CPU 6: hi: 186, btch: 31 usd: 23
Jul 30 19:52:13 localhost kernel: [9623794.888477] CPU 7: hi: 186, btch: 31 usd: 64
Jul 30 19:52:13 localhost kernel: [9623794.888478] Node 0 Normal per-cpu:
Jul 30 19:52:13 localhost kernel: [9623794.888479] CPU 0: hi: 186, btch: 31 usd: 162
Jul 30 19:52:13 localhost kernel: [9623794.888480] CPU 1: hi: 186, btch: 31 usd: 10
Jul 30 19:52:13 localhost kernel: [9623794.888481] CPU 2: hi: 186, btch: 31 usd: 41
Jul 30 19:52:13 localhost kernel: [9623794.888482] CPU 3: hi: 186, btch: 31 usd: 51
Jul 30 19:52:13 localhost kernel: [9623794.888483] CPU 4: hi: 186, btch: 31 usd: 50
Jul 30 19:52:13 localhost kernel: [9623794.888484] CPU 5: hi: 186, btch: 31 usd: 69
Jul 30 19:52:13 localhost kernel: [9623794.888485] CPU 6: hi: 186, btch: 31 usd: 52
Jul 30 19:52:13 localhost kernel: [9623794.888486] CPU 7: hi: 186, btch: 31 usd: 71
Jul 30 19:52:13 localhost kernel: [9623794.888489] active_anon:3986522 inactive_anon:171 isolated_anon:0
Jul 30 19:52:13 localhost kernel: [9623794.888490] active_file:285 inactive_file:1714 isolated_file:0
Jul 30 19:52:13 localhost kernel: [9623794.888490] unevictable:0 dirty:73 writeback:0 unstable:0
Jul 30 19:52:13 localhost kernel: [9623794.888491] free:33382 slab_reclaimable:4700 slab_unreclaimable:7869
Jul 30 19:52:13 localhost kernel: [9623794.888491] mapped:312 shmem:172 pagetables:10759 bounce:0
Jul 30 19:52:13 localhost kernel: [9623794.888492] Node 0 DMA free:15692kB min:60kB low:72kB high:88kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15304kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 30 19:52:13 localhost kernel: [9623794.888497] lowmem_reserve[]: 0 3512 16137 16137
Jul 30 19:52:13 localhost kernel: [9623794.888499] Node 0 DMA32 free:65108kB min:14692kB low:18364kB high:22036kB active_anon:3117388kB inactive_anon:0kB active_file:788kB inactive_file:6920kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3596500kB mlocked:0kB dirty:352kB writeback:0kB mapped:524kB shmem:0kB slab_reclaimable:3104kB slab_unreclaimable:2688kB kernel_stack:128kB pagetables:2292kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1207 all_unreclaimable? yes
Jul 30 19:52:13 localhost kernel: [9623794.888505] lowmem_reserve[]: 0 0 12625 12625
Jul 30 19:52:13 localhost kernel: [9623794.888506] Node 0 Normal free:52728kB min:52824kB low:66028kB high:79236kB active_anon:12828700kB inactive_anon:684kB active_file:352kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:12928000kB mlocked:0kB dirty:0kB writeback:0kB mapped:724kB shmem:688kB slab_reclaimable:15696kB slab_unreclaimable:28788kB kernel_stack:1544kB pagetables:40744kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:416 all_unreclaimable? no
Jul 30 19:52:13 localhost kernel: [9623794.888512] lowmem_reserve[]: 0 0 0 0
Jul 30 19:52:13 localhost kernel: [9623794.888513] Node 0 DMA: 34kB 28kB 116kB 132kB 264kB 1128kB 0256kB 0512kB 11024kB 12048kB 34096kB = 15692kB
Jul 30 19:52:13 localhost kernel: [9623794.888519] Node 0 DMA32: 3874kB 5188kB 34916kB 36432kB 22864kB 88128kB 25256kB 4512kB 01024kB 12048kB 14096kB = 63372kB
Jul 30 19:52:13 localhost kernel: [9623794.888523] Node 0 Normal: 121164kB 268kB 116kB 132kB 164kB 1128kB 1256kB 1512kB 11024kB 12048kB 0*4096kB = 52752kB
Jul 30 19:52:13 localhost kernel: [9623794.888528] 2204 total pagecache pages
Jul 30 19:52:13 localhost kernel: [9623794.888529] 0 pages in swap cache
Jul 30 19:52:13 localhost kernel: [9623794.888530] Swap cache stats: add 0, delete 0, find 0/0
Jul 30 19:52:13 localhost kernel: [9623794.888531] Free swap = 0kB
Jul 30 19:52:13 localhost kernel: [9623794.888532] Total swap = 0kB
Jul 30 19:52:13 localhost kernel: [9623794.912255] 4194303 pages RAM
Jul 30 19:52:13 localhost kernel: [9623794.912256] 111360 pages reserved
Jul 30 19:52:13 localhost kernel: [9623794.912257] 1974 pages shared
Jul 30 19:52:13 localhost kernel: [9623794.912258] 4044257 pages non-shared
Jul 30 19:52:13 localhost kernel: [9623794.912259] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 30 19:52:13 localhost kernel: [9623794.912273] [ 438] 0 438 2670 112 6 -17 -1000 udevd
Jul 30 19:52:13 localhost kernel: [9623794.912276] [ 748] 0 748 1019 22 6 0 0 agetty
Jul 30 19:52:13 localhost kernel: [9623794.912278] [ 1038] 81 1038 5351 58 1 0 0 dbus-daemon
Jul 30 19:52:13 localhost kernel: [9623794.912280] [ 1067] 0 1067 1020 28 0 0 0 acpid
Jul 30 19:52:13 localhost kernel: [9623794.912282] [ 1178] 0 1178 20323 226 6 0 0 master
Jul 30 19:52:13 localhost kernel: [9623794.912283] [ 1191] 89 1191 20386 233 6 0 0 qmgr
Jul 30 19:52:13 localhost kernel: [9623794.912286] [ 1194] 0 1194 5124 163 6 0 0 crond
Jul 30 19:52:13 localhost kernel: [9623794.912287] [ 1207] 0 1207 5385 47 6 0 0 atd
Jul 30 19:52:13 localhost kernel: [9623794.912289] [ 1355] 0 1355 24666 111 3 0 0 sgagent
Jul 30 19:52:13 localhost kernel: [9623794.912291] [ 1593] 0 1593 1016 22 7 0 0 mingetty
Jul 30 19:52:13 localhost kernel: [9623794.912293] [ 1596] 0 1596 1016 22 7 0 0 mingetty
Jul 30 19:52:13 localhost kernel: [9623794.912295] [ 1598] 0 1598 1016 22 3 0 0 mingetty
Jul 30 19:52:13 localhost kernel: [9623794.912298] [ 1600] 0 1600 1016 22 3 0 0 mingetty
Jul 30 19:52:13 localhost kernel: [9623794.912302] [ 1604] 0 1604 2670 117 6 -17 -1000 udevd
Jul 30 19:52:13 localhost kernel: [9623794.912304] [ 1605] 0 1605 2669 110 0 -17 -1000 udevd
Jul 30 19:52:13 localhost kernel: [9623794.912306] [ 1606] 0 1606 1016 22 7 0 0 mingetty
Jul 30 19:52:13 localhost kernel: [9623794.912308] [ 1608] 0 1608 1016 21 3 0 0 mingetty
Jul 30 19:52:13 localhost kernel: [9623794.912310] [ 9286] 500 9286 4504 48 3 0 0 zabbix_agentd
Jul 30 19:52:13 localhost kernel: [9623794.912312] [ 9288] 500 9288 4504 189 6 0 0 zabbix_agentd
Jul 30 19:52:13 localhost kernel: [9623794.912314] [ 9289] 500 9289 4504 72 6 0 0 zabbix_agentd
Jul 30 19:52:13 localhost kernel: [9623794.912316] [ 9290] 500 9290 4504 72 6 0 0 zabbix_agentd
Jul 30 19:52:13 localhost kernel: [9623794.912318] [ 9291] 500 9291 4504 72 6 0 0 zabbix_agentd
Jul 30 19:52:13 localhost kernel: [9623794.912320] [ 9292] 500 9292 4507 59 6 0 0 zabbix_agentd
Jul 30 19:52:13 localhost kernel: [9623794.912322] [27750] 0 27750 23289 76 6 -17 -1000 auditd
Jul 30 19:52:13 localhost kernel: [9623794.912324] [19731] 0 19731 62992 1259 6 0 0 rsyslogd
Jul 30 19:52:13 localhost kernel: [9623794.912326] [24617] 0 24617 16665 178 6 -17 -1000 sshd
Jul 30 19:52:13 localhost kernel: [9623794.912328] [19232] 0 19232 9462 4671 6 0 0 secu-tcs-agent
Jul 30 19:52:13 localhost kernel: [9623794.912330] [19241] 0 19241 768 342 6 0 0 sap1004
Jul 30 19:52:13 localhost kernel: [9623794.912332] [19243] 0 19243 7866 3151 4 0 0 sap1005
Jul 30 19:52:13 localhost kernel: [9623794.912334] [19620] 0 19620 11827 1440 6 0 0 barad_agent
Jul 30 19:52:13 localhost kernel: [9623794.912336] [19628] 0 19628 15060 1839 6 0 0 barad_agent
Jul 30 19:52:13 localhost kernel: [9623794.912338] [19629] 0 19629 109425 2555 6 0 0 barad_agent
Jul 30 19:52:13 localhost kernel: [9623794.912340] [23964] 0 23964 1792 1402 6 0 0 sap1002
Jul 30 19:52:13 localhost kernel: [9623794.912341] [ 4812] 0 4812 9754 3741 6 0 0 sap1009
Jul 30 19:52:13 localhost kernel: [9623794.912344] [18891] 0 18891 2331 92 6 0 0 watchdog.sh
Jul 30 19:52:13 localhost kernel: [9623794.912346] [31004] 89 31004 20347 220 6 0 0 pickup
Jul 30 19:52:13 localhost kernel: [9623794.912348] [ 934] 0 934 1018 21 1 0 0 sleep

[KernelMaker]

你的这种情况不是内存泄漏，就是连接数太多并且请求的数据较大导致的，如果条件允许的话，你可以断开连接观察下内存是否会下降，我们这边也出现过这样的问题，不过机器内存160G，所以影响不大，之前pika线上有过内存占用50G，我们断开所以连接观察，内存迅速回落至8G，还是推荐你减少连接数，1000多连接挂在pika上其实没太大意义

[yanjie108]

好的，�谢谢