Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

mariadb crash after fixes for #206 and #207 #208

Closed
jdcoats opened this issue Mar 11, 2024 · 36 comments
Closed

mariadb crash after fixes for #206 and #207 #208

jdcoats opened this issue Mar 11, 2024 · 36 comments
Assignees
@xmacan

Comments

@jdcoats
Copy link

jdcoats commented Mar 11, 2024 

240309  8:17:22 [ERROR] mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.

To report this bug, see https://mariadb.com/kb/en/reporting-bugs

We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.

Server version: 10.6.14-MariaDB-log source revision: c93754d45e5d9379e3e23d7ada1d5f21d2711f66
key_buffer_size=2097152
read_buffer_size=262144
max_used_connections=498
max_threads=65537
thread_count=201
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 151023922 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x7f11c43e1ea8
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x7f13de286d18 thread_stack 0x49000
/usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x5618c44f071e]
/usr/sbin/mysqld(handle_fatal_signal+0x471)[0x5618c407ac31]
/lib64/libc.so.6(+0x38770)[0x7f1c13e95770]
/usr/sbin/mysqld(+0xcb41c4)[0x5618c43b31c4]
/usr/sbin/mysqld(+0xcb69c7)[0x5618c43b59c7]
/usr/sbin/mysqld(+0xcb6f51)[0x5618c43b5f51]
/usr/sbin/mysqld(+0xc3676e)[0x5618c433576e]
/usr/sbin/mysqld(_Z17mysql_alter_tableP3THDPK25st_mysql_const_lex_stringS3_P14HA_CREATE_INFOP10TABLE_LISTP13Recreate_infoP10Alter_infojP8st_orderbb+0x4a7d)[0x5618c3f0b17d]
/usr/sbin/mysqld(_Z20mysql_recreate_tableP3THDP10TABLE_LISTP13Recreate_infob+0x1f7)[0x5618c3f0bc87]
/usr/sbin/mysqld(+0x874578)[0x5618c3f73578]
/usr/sbin/mysqld(+0x8763d5)[0x5618c3f753d5]
/usr/sbin/mysqld(_ZN22Sql_cmd_optimize_table7executeEP3THD+0x9e)[0x5618c3f7683e]
/usr/sbin/mysqld(_Z21mysql_execute_commandP3THDb+0x9e6)[0x5618c3e65b56]
/usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x197)[0x5618c3e68e07]
/usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcjb+0x1218)[0x5618c3e6a808]
/usr/sbin/mysqld(_Z10do_commandP3THDb+0xee)[0x5618c3e6b5ae]
/usr/sbin/mysqld(_Z11tp_callbackP13TP_connection+0x87)[0x5618c400e837]
/usr/sbin/mysqld(+0xb10b3c)[0x5618c420fb3c]
/usr/sbin/mysqld(+0xba1503)[0x5618c42a0503]
/lib64/libc.so.6(+0x81ef8)[0x7f1c13edeef8]
/lib64/libc.so.6(+0x1015cc)[0x7f1c13f5e5cc]

Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0x7f11c4c48630): OPTIMIZE TABLE mac_track_ips

Connection ID (thread ID): 80183878
Status: NOT_KILLED

Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on,not_null_range_scan=off,hash_join_cardinality=off

The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains
information that should help you find out what is causing the crash.
Writing a core file...
Working directory at /var/lib/mysql
Resource Limits:
Limit                     Soft Limit           Hard Limit           Units
Max cpu time              unlimited            unlimited            seconds
Max file size             unlimited            unlimited            bytes
Max data size             unlimited            unlimited            bytes
Max stack size            8388608              unlimited            bytes
Max core file size        0                    0                    bytes
Max resident set          unlimited            unlimited            bytes
Max processes             402408               402408               processes
Max open files            129031               129031               files
Max locked memory         65536                65536                bytes
Max address space         unlimited            unlimited            bytes
Max file locks            unlimited            unlimited            locks
Max pending signals       402408               402408               signals
Max msgqueue size         819200               819200               bytes
Max nice priority         0                    0
Max realtime priority     0                    0
Max realtime timeout      unlimited            unlimited            us
Core pattern: core

Kernel version: Linux version 5.15.88-gentoo (root@syslog01) (gcc (Gentoo Hardened 11.3.1_p20221209 p3) 11.3.1 20221209, GNU ld (Gentoo 2.38 p4) 2.38) #1 SMP Mon Jan 16 20:09:52 EST 2023
@xmacan
Copy link
Member

xmacan commented Mar 11, 2024

Try to run mysqlcheck

@jdcoats
Copy link
Author

jdcoats commented Mar 11, 2024 •
edited
Loading

all supported checks are okay and yet in the cacti log I still see

2024/03/11 08:36:02 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_resolver.php[158]:db_fetch_assoc(), /lib/database.php[707]:db_fetch_assoc_prepared(), /lib/database.php[727]:db_execute_prepared())
2024/03/11 08:36:02 - CMDPHP ERROR: A DB Row Failed!, Error: MySQL server has gone away
2024/03/11 08:36:02 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_resolver.php[148]:db_fetch_assoc(), /lib/database.php[707]:db_fetch_assoc_prepared(), /lib/database.php[727]:db_execute_prepared())
2024/03/11 08:36:02 - CMDPHP ERROR: A DB Row Failed!, Error: MySQL server has gone away
2024/03/11 08:36:02 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_resolver.php[140]:db_fetch_cell(), /lib/database.php[576]:db_fetch_cell_prepared(), /lib/database.php[598]:db_execute_prepared())
2024/03/11 08:36:02 - CMDPHP ERROR: A DB Cell Failed!, Error: MySQL server has gone away

@jdcoats
Copy link
Author

jdcoats commented Mar 11, 2024 

 # mysqlcheck -A
cacti.aggregate_graph_templates                    OK
cacti.aggregate_graph_templates_graph              OK
cacti.aggregate_graph_templates_item               OK
cacti.aggregate_graphs                             OK
cacti.aggregate_graphs_graph_item                  OK
cacti.aggregate_graphs_items                       OK
cacti.audit_log                                    OK
cacti.automation_devices                           OK
cacti.automation_graph_rule_items                  OK
cacti.automation_graph_rules                       OK
cacti.automation_ips
note     : The storage engine for the table doesn't support check
cacti.automation_match_rule_items                  OK
cacti.automation_networks                          OK
cacti.automation_processes
note     : The storage engine for the table doesn't support check
cacti.automation_snmp                              OK
cacti.automation_snmp_items                        OK
cacti.automation_templates                         OK
cacti.automation_tree_rule_items                   OK
cacti.automation_tree_rules                        OK
cacti.cdef                                         OK
cacti.cdef_items                                   OK
cacti.color_template_items                         OK
cacti.color_templates                              OK
cacti.colors                                       OK
cacti.data_debug                                   OK
cacti.data_input                                   OK
cacti.data_input_data                              OK
cacti.data_input_fields                            OK
cacti.data_local                                   OK
cacti.data_source_profiles                         OK
cacti.data_source_profiles_cf                      OK
cacti.data_source_profiles_rra                     OK
cacti.data_source_purge_action                     OK
cacti.data_source_purge_temp                       OK
cacti.data_source_stats_daily                      OK
cacti.data_source_stats_hourly                     OK
cacti.data_source_stats_hourly_cache
note     : The storage engine for the table doesn't support check
cacti.data_source_stats_hourly_last
note     : The storage engine for the table doesn't support check
cacti.data_source_stats_monthly                    OK
cacti.data_source_stats_weekly                     OK
cacti.data_source_stats_yearly                     OK
cacti.data_template                                OK
cacti.data_template_data                           OK
cacti.data_template_rrd                            OK
cacti.database_stats
note     : The storage engine for the table doesn't support check
cacti.external_links                               OK
cacti.graph_exports                                OK
cacti.graph_exports_tasks                          OK
cacti.graph_local                                  OK
cacti.graph_template_input                         OK
cacti.graph_template_input_defs                    OK
cacti.graph_templates                              OK
cacti.graph_templates_gprint                       OK
cacti.graph_templates_graph                        OK
cacti.graph_templates_item                         OK
cacti.graph_tree                                   OK
cacti.graph_tree_items                             OK
cacti.host                                         OK
cacti.host_graph                                   OK
cacti.host_snmp_cache                              OK
cacti.host_snmp_query                              OK
cacti.host_template                                OK
cacti.host_template_graph                          OK
cacti.host_template_snmp_query                     OK
cacti.mac_track_aggregated_ports                   OK
cacti.mac_track_approved_macs                      OK
cacti.mac_track_arp                                OK
cacti.mac_track_device_types                       OK
cacti.mac_track_devices                            OK
cacti.mac_track_dot1x                              OK
cacti.mac_track_interface_graphs                   OK
cacti.mac_track_interfaces                         OK
cacti.mac_track_ip_ranges                          OK
cacti.mac_track_ips                                OK
cacti.mac_track_macauth                            OK
cacti.mac_track_macwatch                           OK
cacti.mac_track_oui_database                       OK
cacti.mac_track_ports                              OK
cacti.mac_track_processes                          OK
cacti.mac_track_scan_dates                         OK
cacti.mac_track_scanning_functions                 OK
cacti.mac_track_sites                              OK
cacti.mac_track_snmp                               OK
cacti.mac_track_snmp_items                         OK
cacti.mac_track_temp_ports                         OK
cacti.mac_track_vlans                              OK
cacti.notification_queue                           OK
cacti.plugin_config                                OK
cacti.plugin_db_changes                            OK
cacti.plugin_hmib_hrDevices                        OK
cacti.plugin_hmib_hrProcessor                      OK
cacti.plugin_hmib_hrSWInstalled                    OK
cacti.plugin_hmib_hrSWRun                          OK
cacti.plugin_hmib_hrSWRun_ignore                   OK
cacti.plugin_hmib_hrSWRun_last_seen                OK
cacti.plugin_hmib_hrStorage                        OK
cacti.plugin_hmib_hrSystem                         OK
cacti.plugin_hmib_hrSystemTypes                    OK
cacti.plugin_hmib_processes
note     : The storage engine for the table doesn't support check
cacti.plugin_hmib_types                            OK
cacti.plugin_hooks                                 OK
cacti.plugin_intropage_dashboard                   OK
cacti.plugin_intropage_panel_dashboard             OK
cacti.plugin_intropage_panel_data                  OK
cacti.plugin_intropage_panel_definition            OK
cacti.plugin_intropage_trends                      OK
cacti.plugin_intropage_user_auth                   OK
cacti.plugin_monitor_dashboards                    OK
cacti.plugin_monitor_notify_history                OK
cacti.plugin_monitor_reboot_history                OK
cacti.plugin_monitor_uptime                        OK
cacti.plugin_notification_lists                    OK
cacti.plugin_realms                                OK
cacti.plugin_routerconfigs_accounts                OK
cacti.plugin_routerconfigs_backups                 OK
cacti.plugin_routerconfigs_devices                 OK
cacti.plugin_routerconfigs_devicetypes             OK
cacti.plugin_thold_contacts                        OK
cacti.plugin_thold_daemon_data                     OK
cacti.plugin_thold_device_template                 OK
cacti.plugin_thold_host                            OK
cacti.plugin_thold_host_failed                     OK
cacti.plugin_thold_host_template                   OK
cacti.plugin_thold_log                             OK
cacti.plugin_thold_template_contact                OK
cacti.plugin_thold_threshold_contact               OK
cacti.plugin_topx_average                          OK
cacti.plugin_topx_source                           OK
cacti.poller                                       OK
cacti.poller_command                               OK
cacti.poller_data_template_field_mappings          OK
cacti.poller_item                                  OK
cacti.poller_output
note     : The storage engine for the table doesn't support check
cacti.poller_output_boost                          OK
cacti.poller_output_boost_local_data_ids
note     : The storage engine for the table doesn't support check
cacti.poller_output_boost_processes
note     : The storage engine for the table doesn't support check
cacti.poller_output_realtime                       OK
cacti.poller_reindex                               OK
cacti.poller_resource_cache                        OK
cacti.poller_time                                  OK
cacti.processes
note     : The storage engine for the table doesn't support check
cacti.reports                                      OK
cacti.reports_items                                OK
cacti.rrdcheck                                     OK
cacti.sessions                                     OK
cacti.settings                                     OK
cacti.settings_tree                                OK
cacti.settings_user                                OK
cacti.settings_user_group                          OK
cacti.sites                                        OK
cacti.snmp_query                                   OK
cacti.snmp_query_graph                             OK
cacti.snmp_query_graph_rrd                         OK
cacti.snmp_query_graph_rrd_sv                      OK
cacti.snmp_query_graph_sv                          OK
cacti.snmpagent_cache                              OK
cacti.snmpagent_cache_notifications                OK
cacti.snmpagent_cache_textual_conventions          OK
cacti.snmpagent_managers                           OK
cacti.snmpagent_managers_notifications             OK
cacti.snmpagent_mibs                               OK
cacti.snmpagent_notifications_log                  OK
cacti.table_columns                                OK
cacti.table_indexes                                OK
cacti.thold_data                                   OK
cacti.thold_template                               OK
cacti.user_auth                                    OK
cacti.user_auth_cache                              OK
cacti.user_auth_group                              OK
cacti.user_auth_group_members                      OK
cacti.user_auth_group_perms                        OK
cacti.user_auth_group_realm                        OK
cacti.user_auth_perms                              OK
cacti.user_auth_realm                              OK
cacti.user_auth_row_cache                          OK
cacti.user_domains                                 OK
cacti.user_domains_ldap                            OK
cacti.user_log                                     OK
cacti.vdef                                         OK
cacti.vdef_items                                   OK
cacti.version                                      OK
cacti.weathermap_auth                              OK
cacti.weathermap_data                              OK
cacti.weathermap_groups                            OK
cacti.weathermap_maps                              OK
cacti.weathermap_settings                          OK
mysql.column_stats                                 OK
mysql.columns_priv                                 OK
mysql.db                                           OK
mysql.event                                        OK
mysql.func                                         OK
mysql.global_priv                                  OK
mysql.gtid_slave_pos                               OK
mysql.help_category                                OK
mysql.help_keyword                                 OK
mysql.help_relation                                OK
mysql.help_topic                                   OK
mysql.index_stats                                  OK
mysql.innodb_index_stats                           OK
mysql.innodb_table_stats                           OK
mysql.plugin                                       OK
mysql.proc                                         OK
mysql.procs_priv                                   OK
mysql.proxies_priv                                 OK
mysql.roles_mapping                                OK
mysql.servers                                      OK
mysql.table_stats                                  OK
mysql.tables_priv                                  OK
mysql.time_zone                                    OK
mysql.time_zone_leap_second                        OK
mysql.time_zone_name                               OK
mysql.time_zone_transition                         OK
mysql.time_zone_transition_type                    OK
mysql.transaction_registry                         OK
sys.sys_config                                     OK
syslog.syslog                                      OK
syslog.syslog_alert                                OK
syslog.syslog_facilities                           OK
syslog.syslog_host_facilities                      OK
syslog.syslog_hosts                                OK
syslog.syslog_incoming                             OK
syslog.syslog_logs                                 OK
syslog.syslog_priorities                           OK
syslog.syslog_programs                             OK
syslog.syslog_remove                               OK
syslog.syslog_removed                              OK
syslog.syslog_reports                              OK
syslog.syslog_statistics                           OK

@xmacan
Copy link
Member

xmacan commented Mar 11, 2024

check mysql log

@jdcoats
Copy link
Author

jdcoats commented Mar 11, 2024

I did, that's where I got the crash info from this weekend.
This is where i started the db back up this morning.

2024-03-11  8:01:10 0 [Note] Starting MariaDB 10.6.14-MariaDB-log source revision c93754d45e5d9379e3e23d7ada1d5f21d2711f66 as process 5360
2024-03-11  8:01:10 0 [Note] InnoDB: Compressed tables use zlib 1.2.13
2024-03-11  8:01:10 0 [Note] InnoDB: Number of pools: 1
2024-03-11  8:01:10 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
2024-03-11  8:01:10 0 [Note] mysqld: O_TMPFILE is not supported on /tmp/ (disabling future attempts)
2024-03-11  8:01:10 0 [Note] InnoDB: Using liburing
2024-03-11  8:01:10 0 [Note] InnoDB: Initializing buffer pool, total size = 25769803776, chunk size = 134217728
2024-03-11  8:01:10 0 [Note] InnoDB: Completed initialization of buffer pool
2024-03-11  8:01:11 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=126987331300173,126990291476092
2024-03-11  8:01:25 0 [Note] InnoDB: Read redo log up to LSN=126988894530048
2024-03-11  8:01:40 0 [Note] InnoDB: Read redo log up to LSN=126987886193152
2024-03-11  8:01:55 0 [Note] InnoDB: Read redo log up to LSN=126989618440704
2024-03-11  8:01:55 0 [Note] InnoDB: Starting a batch to recover 171586 pages from redo log.
2024-03-11  8:02:10 0 [Note] InnoDB: To recover: 119689 pages from log
2024-03-11  8:02:25 0 [Note] InnoDB: To recover: 110638 pages from log
2024-03-11  8:02:40 0 [Note] InnoDB: To recover: 100518 pages from log
2024-03-11  8:02:55 0 [Note] InnoDB: To recover: 88886 pages from log
2024-03-11  8:03:10 0 [Note] InnoDB: To recover: 76229 pages from log
2024-03-11  8:03:25 0 [Note] InnoDB: To recover: 38871 pages from log
2024-03-11  8:03:36 0 [Note] InnoDB: Starting final batch to recover 75380 pages from redo log.
2024-03-11  8:03:40 0 [Note] InnoDB: To recover: 29953 pages from log
2024-03-11  8:03:55 0 [Note] InnoDB: To recover: 3410 pages from log
2024-03-11  8:03:57 0 [Note] InnoDB: 128 rollback segments are active.
2024-03-11  8:03:57 0 [Note] InnoDB: Removed temporary tablespace data file: "./ibtmp1"
2024-03-11  8:03:57 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2024-03-11  8:03:57 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2024-03-11  8:03:57 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2024-03-11  8:03:57 0 [Note] InnoDB: 10.6.14 started; log sequence number 126990291478829; transaction id 112985414232
2024-03-11  8:03:57 0 [Note] Plugin 'FEEDBACK' is disabled.
2024-03-11  8:03:57 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
2024-03-11  8:03:57 0 [Warning] 'innodb-buffer-pool-instances' was removed. It does nothing now and exists only for compatibility with old my.cnf files.
2024-03-11  8:03:57 0 [Warning] 'innodb-log-files-in-group' was removed. It does nothing now and exists only for compatibility with old my.cnf files.
2024-03-11  8:03:57 0 [Warning] 'innodb-file-format' was removed. It does nothing now and exists only for compatibility with old my.cnf files.
2024-03-11  8:03:57 0 [Warning] 'innodb-large-prefix' was removed. It does nothing now and exists only for compatibility with old my.cnf files.
2024-03-11  8:03:58 0 [Note] DDL_LOG: Crash recovery executed 1 entries
2024-03-11  8:03:58 0 [Note] Server socket created on IP: '127.0.0.1'.
2024-03-11  8:03:58 0 [Note] /usr/sbin/mysqld: ready for connections.
Version: '10.6.14-MariaDB-log'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  Source distribution
2024-03-11  8:04:00 0 [Note] InnoDB: Buffer pool(s) load completed at 240311  8:04:00

@jdcoats
Copy link
Author

jdcoats commented Mar 12, 2024 •
edited
Loading

follow-up confirming each mactrack run produces a stream of this in the log until run is complete :(

2024/03/12 08:08:43 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_resolver.php[158]:db_fetch_assoc(), /lib/database.php[707]:db_fetch_assoc_prepared(), /lib/database.php[727]:db_execute_prepared())
2024/03/12 08:08:43 - CMDPHP ERROR: A DB Row Failed!, Error: MySQL server has gone away
2024/03/12 08:08:43 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_resolver.php[148]:db_fetch_assoc(), /lib/database.php[707]:db_fetch_assoc_prepared(), /lib/database.php[727]:db_execute_prepared())
2024/03/12 08:08:43 - CMDPHP ERROR: A DB Row Failed!, Error: MySQL server has gone away
2024/03/12 08:08:43 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_resolver.php[140]:db_fetch_cell(), /lib/database.php[576]:db_fetch_cell_prepared(), /lib/database.php[598]:db_execute_prepared())
2024/03/12 08:08:43 - CMDPHP ERROR: A DB Cell Failed!, Error: MySQL server has gone away
2024/03/12 08:08:40 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_resolver.php[158]:db_fetch_assoc(), /lib/database.php[707]:db_fetch_assoc_prepared(), /lib/database.php[727]:db_execute_prepared())
2024/03/12 08:08:40 - CMDPHP ERROR: A DB Row Failed!, Error: MySQL server has gone away
2024/03/12 08:08:40 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_resolver.php[148]:db_fetch_assoc(), /lib/database.php[707]:db_fetch_assoc_prepared(), /lib/database.php[727]:db_execute_prepared())
2024/03/12 08:08:40 - CMDPHP ERROR: A DB Row Failed!, Error: MySQL server has gone away
2024/03/12 08:08:40 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_resolver.php[140]:db_fetch_cell(), /lib/database.php[576]:db_fetch_cell_prepared(), /lib/database.php[598]:db_execute_prepared())
2024/03/12 08:08:40 - CMDPHP ERROR: A DB Cell Failed!, Error: MySQL server has gone away

@xmacan
Copy link
Member

xmacan commented Mar 12, 2024

try again:
show variables like "max_connections";
show status like 'max_used_connections';

@xmacan
Copy link
Member

xmacan commented Mar 12, 2024

we need mysql log from problematic time:
2024/03/12 08:08:40 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_resolver.php[140]:db_fetch_cell(), /lib/database.php[576]:db_fetch_cell_prepared(), /lib/database.php[598]:db_execute_prepared())
2024/03/12 08:08:40 - CMDPHP ERROR: A DB Cell Failed!, Error: MySQL server has gone away

@jdcoats
Copy link
Author

jdcoats commented Mar 12, 2024

nothing in mysql log only slow queries, mysql is not crashed or "gone away" cacti is still running fine.

MariaDB [(none)]> show variables like "max_connections";
show status like 'max_used_connections';
+-----------------+-------+
| Variable_name   | Value |
+-----------------+-------+
| max_connections | 1000  |
+-----------------+-------+
1 row in set (0.000 sec)

+----------------------+-------+
| Variable_name        | Value |
+----------------------+-------+
| Max_used_connections | 503   |
+----------------------+-------+
1 row in set (0.000 sec)

INSERT INTO data_source_stats_hourly (local_data_id, rrd_name, average, peak) SELECT local_data_id, rrd_name, AVG(`value`), MAX(`value`) FROM data_source_stats_hourly_cache WHERE `value` IS NOT NULL GROUP BY local_data_id, rrd_name ON DUPLICATE KEY UPDATE average=VALUES(average), peak=VALUES(peak);
# Time: 240312  8:13:48
# User@Host: cacti123[cacti123] @ localhost []
# Thread_id: 1159637  Schema: cacti  QC_hit: No
# Query_time: 7.690063  Lock_time: 0.000046  Rows_sent: 0  Rows_examined: 6975956
# Rows_affected: 153248  Bytes_sent: 62
SET timestamp=1710245628;
INSERT INTO data_source_stats_hourly (local_data_id, rrd_name, average, peak) SELECT local_data_id, rrd_name, AVG(`value`), MAX(`value`) FROM data_source_stats_hourly_cache WHERE `value` IS NOT NULL GROUP BY local_data_id, rrd_name ON DUPLICATE KEY UPDATE average=VALUES(average), peak=VALUES(peak);
# Time: 240312  8:14:39
# User@Host: cacti123[cacti123] @ localhost []
# Thread_id: 1160328  Schema: cacti  QC_hit: No
# Query_time: 5.032315  Lock_time: 0.000193  Rows_sent: 45  Rows_examined: 72070
# Rows_affected: 0  Bytes_sent: 8806
SET timestamp=1710245679;
SELECT site_name, device_id, device_name, hostname, mtp.mac_address, vendor_name, ip_address, dns_hostname, port_number, port_name, vlan_id, vlan_name, MAX(scan_date) AS scan_date, mtm.mac_id FROM mac_track_ports AS mtp LEFT JOIN mac_track_sites AS mts ON mtp.site_id = mts.site_id LEFT JOIN mac_track_macauth AS mtm ON mtm.mac_address = mtp.mac_address LEFT JOIN mac_track_oui_database AS mtod ON mtod.vendor_mac = mtp.vendor_mac WHERE mtp.device_id = 287 GROUP BY device_id, mtp.mac_address, port_number, ip_address ORDER BY `device_name` ASC LIMIT 0,45;
# Time: 240312  8:14:44
# User@Host: cacti123[cacti123] @ localhost []
# Thread_id: 1160331  Schema: cacti  QC_hit: No
# Query_time: 8.353788  Lock_time: 0.000065  Rows_sent: 0  Rows_examined: 6975973
# Rows_affected: 153168  Bytes_sent: 62
SET timestamp=1710245684;
INSERT INTO data_source_stats_hourly (local_data_id, rrd_name, average, peak) SELECT local_data_id, rrd_name, AVG(`value`), MAX(`value`) FROM data_source_stats_hourly_cache WHERE `value` IS NOT NULL GROUP BY local_data_id, rrd_name ON DUPLICATE KEY UPDATE average=VALUES(average), peak=VALUES(peak);
# Time: 240312  8:15:33
# User@Host: cacti123[cacti123] @ localhost []
# Thread_id: 1161265  Schema: cacti  QC_hit: No
# Query_time: 6.660760  Lock_time: 0.000102  Rows_sent: 45  Rows_examined: 443599
# Rows_affected: 0  Bytes_sent: 9048
SET timestamp=1710245733;
SELECT site_name, device_id, device_name, hostname, mtp.mac_address, vendor_name, ip_address, dns_hostname, port_number, port_name, vlan_id, vlan_name, MAX(scan_date) AS scan_date, mtm.mac_id FROM mac_track_ports AS mtp LEFT JOIN mac_track_sites AS mts ON mtp.site_id = mts.site_id LEFT JOIN mac_track_macauth AS mtm ON mtm.mac_address = mtp.mac_address LEFT JOIN mac_track_oui_database AS mtod ON mtod.vendor_mac = mtp.vendor_mac WHERE mtp.device_id = 31 GROUP BY device_id, mtp.mac_address, port_number, ip_address ORDER BY `device_name` ASC LIMIT 0,45;
# Time: 240312  8:15:53
# User@Host: cacti123[cacti123] @ localhost []
# Thread_id: 1161483  Schema: cacti  QC_hit: No
# Query_time: 8.042795  Lock_time: 0.000055  Rows_sent: 0  Rows_examined: 6975972
# Rows_affected: 153426  Bytes_sent: 62
SET timestamp=1710245753;
INSERT INTO data_source_stats_hourly (local_data_id, rrd_name, average, peak) SELECT local_data_id, rrd_name, AVG(`value`), MAX(`value`) FROM data_source_stats_hourly_cache WHERE `value` IS NOT NULL GROUP BY local_data_id, rrd_name ON DUPLICATE KEY UPDATE average=VALUES(average), peak=VALUES(peak);
# Time: 240312  8:16:49
# User@Host: cacti123[cacti123] @ localhost []
# Thread_id: 1162672  Schema: cacti  QC_hit: No
# Query_time: 8.384592  Lock_time: 0.000061  Rows_sent: 0  Rows_examined: 6975989
# Rows_affected: 152943  Bytes_sent: 62
SET timestamp=1710245809;
INSERT INTO data_source_stats_hourly (local_data_id, rrd_name, average, peak) SELECT local_data_id, rrd_name, AVG(`value`), MAX(`value`) FROM data_source_stats_hourly_cache WHERE `value` IS NOT NULL GROUP BY local_data_id, rrd_name ON DUPLICATE KEY UPDATE average=VALUES(average), peak=VALUES(peak);
# Time: 240312  8:17:47
# User@Host: cacti123[cacti123] @ localhost []
# Thread_id: 1163691  Schema: cacti  QC_hit: No
# Query_time: 7.779472  Lock_time: 0.000053  Rows_sent: 0  Rows_examined: 6975988
# Rows_affected: 153260  Bytes_sent: 62
SET timestamp=1710245867;

@xmacan
Copy link
Member

xmacan commented Mar 12, 2024

try to run manually:
php mactrack_resolver.php -d
Is it working?

You can try edit mactrack_resolver.php, line 140:
while (1) {
Change it to
while (1) {
usleep(10000);

It makes small break in each cycle process

@jdcoats
Copy link
Author

jdcoats commented Mar 12, 2024 

 # php mactrack_resolver.php -d
2024/03/12 09:09:54 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:09:57 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:10:00 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:10:03 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:10:06 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:10:09 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:10:12 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:10:15 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:10:18 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:10:21 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:10:24 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:10:27 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:10:30 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:10:33 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:10:36 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:10:39 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:10:42 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:10:45 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:10:48 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:10:51 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:10:54 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:10:57 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:11:00 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:11:03 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:11:06 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:11:09 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:11:12 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:11:15 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:11:18 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:11:21 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:11:24 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:11:27 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:11:30 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:11:33 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:11:36 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:11:39 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:11:42 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:11:45 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:11:48 - MACTRACK DEBUG: No IP's require resolving this pass
2024/03/12 09:11:51 - MACTRACK DEBUG: No IP's require resolving this pass

@xmacan
Copy link
Member

xmacan commented Mar 12, 2024

is it endless?

@jdcoats
Copy link
Author

jdcoats commented Mar 12, 2024

yes i believe so, I had just stopped it but I can let it go a while to confirm

@xmacan
Copy link
Member

xmacan commented Mar 12, 2024

oki, I have a lot of information ;-) I will look at it

@xmacan xmacan self-assigned this Mar 12, 2024
@jdcoats
Copy link
Author

jdcoats commented Mar 12, 2024

it did run endless for over 20min

@xmacan
Copy link
Member

xmacan commented Mar 12, 2024

last question, show me result of:
SELECT last_rundate, COUNT(last_rundate) AS devices FROM mac_track_devices WHERE disabled = '' GROUP BY last_rundate ORDER BY last_rundate DESC;

@jdcoats
Copy link
Author

jdcoats commented Mar 12, 2024 

MariaDB [cacti]> SELECT last_rundate, COUNT(last_rundate) AS devices FROM mac_track_devices WHERE disabled = '' GROUP BY last_rundate ORDER BY last_rundate DESC;
+---------------------+---------+
| last_rundate        | devices |
+---------------------+---------+
| 2024-03-12 08:08:59 |       1 |
| 2024-03-12 08:08:56 |       3 |
| 2024-03-12 08:08:54 |       4 |
| 2024-03-12 08:08:52 |       7 |
| 2024-03-12 08:08:50 |       8 |
| 2024-03-12 08:08:48 |       6 |
| 2024-03-12 08:08:46 |       6 |
| 2024-03-12 08:08:45 |       1 |
| 2024-03-12 08:08:44 |       2 |
| 2024-03-12 08:08:43 |       1 |
| 2024-03-12 08:08:42 |       5 |
| 2024-03-12 08:08:40 |       3 |
| 2024-03-12 08:08:38 |       3 |
| 2024-03-12 08:08:37 |       1 |
| 2024-03-12 08:08:36 |       6 |
| 2024-03-12 08:08:34 |       2 |
| 2024-03-12 08:08:32 |       2 |
| 2024-03-12 08:08:30 |       7 |
| 2024-03-12 08:08:28 |       7 |
| 2024-03-12 08:08:26 |       4 |
| 2024-03-12 08:08:25 |       1 |
| 2024-03-12 08:08:24 |       1 |
| 2024-03-12 08:08:19 |       1 |
| 2024-03-12 08:08:18 |       1 |
| 2024-03-12 08:08:13 |       4 |
| 2024-03-12 08:08:07 |       1 |
| 2024-03-12 08:08:05 |       1 |
| 2024-03-12 08:07:59 |       1 |
| 2024-03-12 08:07:57 |       1 |
| 2024-03-12 08:07:53 |       1 |
| 2024-03-12 08:07:49 |       1 |
| 2024-03-12 08:07:46 |       1 |
| 2024-03-12 08:07:45 |       1 |
| 2024-03-12 08:07:43 |       2 |
| 2024-03-12 08:07:39 |       1 |
| 2024-03-12 08:07:37 |       9 |
| 2024-03-12 08:07:36 |       1 |
| 2024-03-12 08:07:35 |       3 |
| 2024-03-12 08:07:34 |       1 |
| 2024-03-12 08:07:33 |       4 |
| 2024-03-12 08:07:31 |       1 |
| 2024-03-12 08:07:28 |       1 |
| 2024-03-12 08:07:27 |       2 |
| 2024-03-12 08:07:25 |       4 |
| 2024-03-12 08:07:23 |       1 |
| 2024-03-12 08:07:21 |       3 |
| 2024-03-12 08:07:19 |       3 |
| 2024-03-12 08:07:17 |       4 |
| 2024-03-12 08:07:15 |       3 |
| 2024-03-12 08:07:13 |       3 |
| 2024-03-12 08:07:12 |       1 |
| 2024-03-12 08:07:11 |       9 |
| 2024-03-12 08:07:09 |      14 |
| 2024-03-12 08:07:06 |       6 |
| 2024-03-12 08:07:05 |      20 |
| 2024-03-12 08:07:04 |      24 |
+---------------------+---------+
56 rows in set (0.001 sec)

@xmacan
Copy link
Member

xmacan commented Mar 12, 2024

Script is trying find unresolved IP addresses. If there isn't any unresolved, it is sleeping for 3 seconds and try it again. It is repeated for 20 minutes. The most time scripts is "waiting" for mactrack_scanner.php finish his work. mactrack_scanner.php can add new unresolved IPs. So it is normal behavior.

From my perspective - here is strange logic but it works.

@xmacan
Copy link
Member

xmacan commented Mar 12, 2024

try reinstall/recompile/update your mariadb

@xmacan
Copy link
Member

xmacan commented Mar 12, 2024

@jdcoats how is your mactrack setup:
Scanning Frequency?
Concurrent Processes?
Scanner Max Runtime?
Number of devices?

@jdcoats
Copy link
Author

jdcoats commented Mar 12, 2024

Scanning Frequency 4hr
Concurrent Processes? 50
Scanner Max Runtime? 20
Number of devices? 236

@xmacan
Copy link
Member

xmacan commented Mar 20, 2024

@jdcoats any news here? I created small fix for dns_resolver correct process end. You can test it but I think that you have problem with db server.

@jdcoats
Copy link
Author

jdcoats commented Mar 20, 2024

I've disabled mactrack completely because it took down cacti twice in 2 weeks.

@TheWitness
Copy link
Member

Ugh!

@jdcoats
Copy link
Author

jdcoats commented Apr 17, 2024

I re-enabled to see if I could get anything new but I still get this when the mactrack poller is running.

2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[237]:db_process_remove(), /plugins/mactrack/lib/mactrack_functions.php[2209]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[236]:db_update_device_status(), /plugins/mactrack/lib/mactrack_functions.php[2240]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[195]:get_standard_arp_table(), /plugins/mactrack/lib/mactrack_functions.php[477]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[195]:get_standard_arp_table(), /plugins/mactrack/lib/mactrack_functions.php[466]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[195]:get_standard_arp_table(), /plugins/mactrack/lib/mactrack_functions.php[466]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[195]:get_standard_arp_table(), /plugins/mactrack/lib/mactrack_functions.php[466]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[195]:get_standard_arp_table(), /plugins/mactrack/lib/mactrack_functions.php[466]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[195]:get_standard_arp_table(), /plugins/mactrack/lib/mactrack_functions.php[466]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[195]:get_standard_arp_table(), /plugins/mactrack/lib/mactrack_functions.php[466]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[195]:get_standard_arp_table(), /plugins/mactrack/lib/mactrack_functions.php[466]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[195]:get_standard_arp_table(), /plugins/mactrack/lib/mactrack_functions.php[466]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[195]:get_standard_arp_table(), /plugins/mactrack/lib/mactrack_functions.php[466]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[195]:get_standard_arp_table(), /plugins/mactrack/lib/mactrack_functions.php[466]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[195]:get_standard_arp_table(), /plugins/mactrack/lib/mactrack_functions.php[466]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[195]:get_standard_arp_table(), /plugins/mactrack/lib/mactrack_functions.php[466]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[195]:get_standard_arp_table(), /plugins/mactrack/lib/mactrack_functions.php[466]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[195]:get_standard_arp_table(), /plugins/mactrack/lib/mactrack_functions.php[466]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[195]:get_standard_arp_table(), /plugins/mactrack/lib/mactrack_functions.php[466]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[195]:get_standard_arp_table(), /plugins/mactrack/lib/mactrack_functions.php[466]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[171]:get_IOS_dot1dTpFdbEntry_ports(), /plugins/mactrack/lib/mactrack_cisco.php[483]:db_store_device_port_results(), /plugins/mactrack/lib/mactrack_functions.php[2302]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[171]:get_IOS_dot1dTpFdbEntry_ports(), /plugins/mactrack/lib/mactrack_cisco.php[483]:db_store_device_port_results(), /plugins/mactrack/lib/mactrack_functions.php[2274]:db_check_auth(), /plugins/mactrack/lib/mactrack_functions.php[2317]:db_fetch_cell_prepared(), /lib/database.php[598]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[171]:get_IOS_dot1dTpFdbEntry_ports(), /plugins/mactrack/lib/mactrack_cisco.php[483]:db_store_device_port_results(), /plugins/mactrack/lib/mactrack_functions.php[2302]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[171]:get_IOS_dot1dTpFdbEntry_ports(), /plugins/mactrack/lib/mactrack_cisco.php[483]:db_store_device_port_results(), /plugins/mactrack/lib/mactrack_functions.php[2274]:db_check_auth(), /plugins/mactrack/lib/mactrack_functions.php[2317]:db_fetch_cell_prepared(), /lib/database.php[598]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[171]:get_IOS_dot1dTpFdbEntry_ports(), /plugins/mactrack/lib/mactrack_cisco.php[483]:db_store_device_port_results(), /plugins/mactrack/lib/mactrack_functions.php[2302]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[171]:get_IOS_dot1dTpFdbEntry_ports(), /plugins/mactrack/lib/mactrack_cisco.php[483]:db_store_device_port_results(), /plugins/mactrack/lib/mactrack_functions.php[2274]:db_check_auth(), /plugins/mactrack/lib/mactrack_functions.php[2317]:db_fetch_cell_prepared(), /lib/database.php[598]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[171]:get_IOS_dot1dTpFdbEntry_ports(), /plugins/mactrack/lib/mactrack_cisco.php[483]:db_store_device_port_results(), /plugins/mactrack/lib/mactrack_functions.php[2302]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[171]:get_IOS_dot1dTpFdbEntry_ports(), /plugins/mactrack/lib/mactrack_cisco.php[483]:db_store_device_port_results(), /plugins/mactrack/lib/mactrack_functions.php[2274]:db_check_auth(), /plugins/mactrack/lib/mactrack_functions.php[2317]:db_fetch_cell_prepared(), /lib/database.php[598]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[171]:get_IOS_dot1dTpFdbEntry_ports(), /plugins/mactrack/lib/mactrack_cisco.php[483]:db_store_device_port_results(), /plugins/mactrack/lib/mactrack_functions.php[2302]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[171]:get_IOS_dot1dTpFdbEntry_ports(), /plugins/mactrack/lib/mactrack_cisco.php[483]:db_store_device_port_results(), /plugins/mactrack/lib/mactrack_functions.php[2274]:db_check_auth(), /plugins/mactrack/lib/mactrack_functions.php[2317]:db_fetch_cell_prepared(), /lib/database.php[598]:db_execute_prepared())

@xmacan
Copy link
Member

xmacan commented Apr 18, 2024

@jdcoats is it with last develop code? What php version?

@jdcoats
Copy link
Author

jdcoats commented Apr 18, 2024

yes latest develop + your last pull request :) php 8.1.20

@xmacan
Copy link
Member

xmacan commented Apr 18, 2024

is it started automatic from poller or manually via php poller_mactrack.php?

@jdcoats
Copy link
Author

jdcoats commented Apr 18, 2024

automatic

@xmacan
Copy link
Member

xmacan commented Apr 18, 2024

are you using more mactrack sites?

@jdcoats
Copy link
Author

jdcoats commented Apr 18, 2024

see my sites examples posted yesterday in #204

@xmacan
Copy link
Member

xmacan commented Apr 19, 2024

I re-enabled to see if I could get anything new but I still get this when the mactrack poller is running.

2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[237]:db_process_remove(), /plugins/mactrack/lib/mactrack_functions.php[2209]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[236]:db_update_device_status(), /plugins/mactrack/lib/mactrack_functions.php[2240]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[195]:get_standard_arp_table(), /plugins/mactrack/lib/mactrack_functions.php[477]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[195]:get_standard_arp_table(), /plugins/mactrack/lib/mactrack_functions.php[466]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[195]:get_standard_arp_table(), /plugins/mactrack/lib/mactrack_functions.php[466]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[195]:get_standard_arp_table(), /plugins/mactrack/lib/mactrack_functions.php[466]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[195]:get_standard_arp_table(), /plugins/mactrack/lib/mactrack_functions.php[466]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[195]:get_standard_arp_table(), /plugins/mactrack/lib/mactrack_functions.php[466]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[195]:get_standard_arp_table(), /plugins/mactrack/lib/mactrack_functions.php[466]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[195]:get_standard_arp_table(), /plugins/mactrack/lib/mactrack_functions.php[466]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[195]:get_standard_arp_table(), /plugins/mactrack/lib/mactrack_functions.php[466]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[195]:get_standard_arp_table(), /plugins/mactrack/lib/mactrack_functions.php[466]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[195]:get_standard_arp_table(), /plugins/mactrack/lib/mactrack_functions.php[466]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[195]:get_standard_arp_table(), /plugins/mactrack/lib/mactrack_functions.php[466]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[195]:get_standard_arp_table(), /plugins/mactrack/lib/mactrack_functions.php[466]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[195]:get_standard_arp_table(), /plugins/mactrack/lib/mactrack_functions.php[466]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[195]:get_standard_arp_table(), /plugins/mactrack/lib/mactrack_functions.php[466]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[195]:get_standard_arp_table(), /plugins/mactrack/lib/mactrack_functions.php[466]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[195]:get_standard_arp_table(), /plugins/mactrack/lib/mactrack_functions.php[466]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[171]:get_IOS_dot1dTpFdbEntry_ports(), /plugins/mactrack/lib/mactrack_cisco.php[483]:db_store_device_port_results(), /plugins/mactrack/lib/mactrack_functions.php[2302]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[171]:get_IOS_dot1dTpFdbEntry_ports(), /plugins/mactrack/lib/mactrack_cisco.php[483]:db_store_device_port_results(), /plugins/mactrack/lib/mactrack_functions.php[2274]:db_check_auth(), /plugins/mactrack/lib/mactrack_functions.php[2317]:db_fetch_cell_prepared(), /lib/database.php[598]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[171]:get_IOS_dot1dTpFdbEntry_ports(), /plugins/mactrack/lib/mactrack_cisco.php[483]:db_store_device_port_results(), /plugins/mactrack/lib/mactrack_functions.php[2302]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[171]:get_IOS_dot1dTpFdbEntry_ports(), /plugins/mactrack/lib/mactrack_cisco.php[483]:db_store_device_port_results(), /plugins/mactrack/lib/mactrack_functions.php[2274]:db_check_auth(), /plugins/mactrack/lib/mactrack_functions.php[2317]:db_fetch_cell_prepared(), /lib/database.php[598]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[171]:get_IOS_dot1dTpFdbEntry_ports(), /plugins/mactrack/lib/mactrack_cisco.php[483]:db_store_device_port_results(), /plugins/mactrack/lib/mactrack_functions.php[2302]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[171]:get_IOS_dot1dTpFdbEntry_ports(), /plugins/mactrack/lib/mactrack_cisco.php[483]:db_store_device_port_results(), /plugins/mactrack/lib/mactrack_functions.php[2274]:db_check_auth(), /plugins/mactrack/lib/mactrack_functions.php[2317]:db_fetch_cell_prepared(), /lib/database.php[598]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[171]:get_IOS_dot1dTpFdbEntry_ports(), /plugins/mactrack/lib/mactrack_cisco.php[483]:db_store_device_port_results(), /plugins/mactrack/lib/mactrack_functions.php[2302]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[171]:get_IOS_dot1dTpFdbEntry_ports(), /plugins/mactrack/lib/mactrack_cisco.php[483]:db_store_device_port_results(), /plugins/mactrack/lib/mactrack_functions.php[2274]:db_check_auth(), /plugins/mactrack/lib/mactrack_functions.php[2317]:db_fetch_cell_prepared(), /lib/database.php[598]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[171]:get_IOS_dot1dTpFdbEntry_ports(), /plugins/mactrack/lib/mactrack_cisco.php[483]:db_store_device_port_results(), /plugins/mactrack/lib/mactrack_functions.php[2302]:db_execute_prepared())
2024/04/17 16:09:48 - CMDPHP SQL Backtrace: (/plugins/mactrack/mactrack_scanner.php[171]:get_IOS_dot1dTpFdbEntry_ports(), /plugins/mactrack/lib/mactrack_cisco.php[483]:db_store_device_port_results(), /plugins/mactrack/lib/mactrack_functions.php[2274]:db_check_auth(), /plugins/mactrack/lib/mactrack_functions.php[2317]:db_fetch_cell_prepared(), /lib/database.php[598]:db_execute_prepared())

I think that my last change fixes this too. @jdcoats could you test it, please?

@jdcoats
Copy link
Author

jdcoats commented Apr 19, 2024

yes, it has not crashed since i re-enabled it so if it makes it through the weekend id says something since then has fixed it.

@TheWitness
Copy link
Member

Enjoyed watching you two work through this. Great work!

@jdcoats
Copy link
Author

jdcoats commented Apr 22, 2024

this is resolved also.

@TheWitness
Copy link
Member

Cool.

@xmacan xmacan closed this as completed Apr 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@xmacan xmacan

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@TheWitness @xmacan @jdcoats