timespan to timedelta dataframe conversion incorrect #99
Comments
|
Just bumping this with a screenshot as a demo of the issue. As a workaround, converting to a csv and then into a dataframe avoids this problem.
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Code Sample, a copy-pastable example if possible
Problem description
When the timestamp has sub-second precision as per the above code example's first table row, it causes the conversion to a pandas data-frame to have an incorrect timedelta. E.g. it resulted in
38 days 21:00:27instead of the correct detla of1 days 0:00:00.32724700. When run in Log Analytics, the timespan value returned was1.00:00:00.3272427. So it seems like there is some timespan -> timedelta conversion issue when sub-second precision occurs.Interestingly, this problem does not manifest if the timespan did not inlcude sub-second precision values as per the second row in the example table.
If query related, does it happen on other platforms (Kusto Web UI, Kusto Explorer, Log Analytics in Azure Monitor)?
No, Log Analytics returns expected timespan values.
Output of
pip freezeand%kql --bugreport[bugreport must be executed immediatly after bug noticed]
{'kqlmagic': {'version': '0.1.114.post22'}, 'platform': {'system': 'Linux', 'release': '5.15.0-58-generic'}, 'packages': {'python-dateutil': '2.8.2', 'traitlets': '5.4.0', 'prettytable': '3.4.1', 'requests': '2.27.1', 'msal': '1.19.0', 'azure-identity': '1.11.0', 'pandas': '1.5.2', 'ipython': '8.5.0', 'ipykernel': '6.15.3', 'pygments': '2.13.0', 'pyperclip': '1.8.2', 'azure-common': '1.1.28', 'msrestazure': '0.6.4', 'psutil': '5.9.2', 'matplotlib': '3.5.3', 'setuptools': '65.3.0', 'plotly': '5.10.0', 'flask': '2.2.2', 'isodate': '0.6.1', 'markdown': '3.4.1', 'beautifulsoup4': '4.11.1', 'lxml': '4.9.1', 'ipywidgets': '7.7.2', 'cryptography': '36.0.1', 'password_strength': '0.0.3.post2'}, 'python': {'name': 'CPython', 'version': '3.10.9', 'branch': ''}, 'kqlmagic_default_options': {'add_kql_ref_to_help': True, 'add_schema_to_help': True, 'allow_py_comments_before_cell': True, 'allow_single_line_cell': True, 'assign_var': None, 'auth_token_warnings': False, 'auth_use_http_client': False, 'auto_dataframe': False, 'auto_limit': 1000000, 'auto_popup_schema': True, 'cache': None, 'cache_folder_name': 'cache_files', 'check_magic_version': True, 'cloud': 'public', 'code_auth_interactive_mode': 'device_code', 'columns_to_local_vars': False, 'cursor_var': None, 'debug': False, 'device_code_login_notification': 'button', 'device_code_notification_email': '', 'display_limit': 100, 'dsn_filename': 'odbc.ini', 'dynamic_to_dataframe': 'object', 'enable_add_items_to_help': True, 'enable_curly_brackets_params': False, 'enable_sso': False, 'enable_suppress_result': True, 'export_folder_name': 'exported_files', 'extras_require': 'default', 'feedback': True, 'is_kernel_intializtion': False, 'is_magic': True, 'json_display': 'raw', 'kernel_id': '***', 'kernel_location': 'local', 'kqlmagic_kernel': False, 'last_raw_result_var': '_kql_raw_result_', 'notebook_app': 'visualstudiocode', 'notebook_service_address': None, 'palette_colors': 10, 'palette_desaturation': 1.0, 'palette_name': 'tab10', 'plot_package': 'plotly', 'plotly_config': None, 'plotly_fs_includejs': False, 'plotly_layout': None, 'popup_interaction': 'auto', 'prettytable_style': 'DEFAULT', 'query_link_destination': 'Kusto.WebExplorer', 'request_app_tag': None, 'request_cache_max_age': 0, 'request_id_tag': None, 'request_user_agent_tag': None, 'request_user_tag': None, 'schema_json_display': 'raw', 'short_errors': True, 'show_conn_info': 'current', 'show_init_banner': False, 'show_query': False, 'show_query_link': False, 'show_query_time': True, 'show_what_new': True, 'sso_db_gc_interval': 168, 'table_package': 'prettytable', 'temp_files_server': 'auto', 'temp_files_server_address': 'http://127.0.0.1:47111', 'temp_folder_location': 'user_dir', 'temp_folder_name': 'temp_files', 'test_notebook_app': 'none', 'timeout': 120, 'try_azcli_login': False, 'try_azcli_login_by_profile': False, 'try_azcli_login_subscription': None, 'try_kqlmaic_sso': False, 'try_msi': None, 'try_token': None, 'try_vscode_login': False, 'use_cache': None, 'validate_connection_string': True, 'warn_missing_dependencies': True, 'warn_missing_env_variables': True}, 'kqlmagic_default_env': {'KQLMAGIC_CONFIGURATION': '\'show_init_banner=False;device_code_login_notification="button";timeout=120;auto_limit=1000000;display_limit=100;json_display="raw";schema_json_display="raw"\''} }[paste the output of
pip freezehere below this line]The text was updated successfully, but these errors were encountered: