Failing to install on 4.0 #544

deanfourie1 · 2024-07-18T11:11:22Z

deanfourie1
Jul 18, 2024

NetBox version

v4.0.7

Topology Views version

v4.0.0

Steps to Reproduce

cd /opt/netbox
Active venv >> source /opt/netbox/venv/bin/activate
install with >> pip install netbox-topology-views
add to local_requirements.txt >> cd /opt/netbox >> echo netbox-topology-views >> local_requirements.txt
add to configuration.py >> cd /opt/netbox/netbox/netbox >> nano configuration.py >> add "netbox_topology_views",
cd /opt/netbox/netbox
pip3 install netbox-topology-views
python3 manage.py migrate netbox_topology_views
python3 manage.py collectstatic --no-input

Added the following to configuration.py

PLUGINS_CONFIG = {
    'netbox_topology_views': {
        'static_image_directory': 'netbox_topology_views/img',
        'allow_coordinates_saving': True,
        'always_save_coordinates': True
    }
}

cd /opt/netbox
./upgrade.sh

Expected Behavior

For ./upgrad.sh to complete, reboot netbox and netbox-rq and be able to use netbox-typology.

Observed Behavior

venv) root@netbox-local:/opt/netbox/netbox# python3 manage.py migrate netbox_topology_views
/opt/netbox/venv/lib/python3.11/site-packages/django/db/backends/utils.py:98: RuntimeWarning: Accessing the database during app initialization is discouraged. To fix this warning, avoid executing queries in AppConfig.ready() or when your app modules are imported.
  warnings.warn(self.APPS_NOT_READY_WARNING_MSG, category=RuntimeWarning)
Operations to perform:
  Apply all migrations: netbox_topology_views
Running migrations:
  No migrations to apply.
(venv) root@netbox-local:/opt/netbox/netbox#

venv) root@netbox-local:/opt/netbox/netbox# python3 manage.py collectstatic --no-input
/opt/netbox/venv/lib/python3.11/site-packages/django/db/backends/utils.py:98: RuntimeWarning: Accessing the database during app initialization is discouraged. To fix this warning, avoid executing queries in AppConfig.ready() or when your app modules are imported.
  warnings.warn(self.APPS_NOT_READY_WARNING_MSG, category=RuntimeWarning)

0 static files copied to '/opt/netbox/netbox/static', 544 unmodified.
(venv) root@netbox-local:/opt/netbox/netbox#

  netbox_topology_views.coordinategroup... Traceback (most recent call last):
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 105, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/psycopg/server_cursor.py", line 294, in execute
    raise ex.with_traceback(None)
psycopg.errors.UndefinedTable: relation "netbox_topology_views_coordinategroup" does not exist
LINE 1: ...opology_views_coordinategroup"."description" FROM "netbox_to...
                                                             ^

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/netbox/netbox/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/management/base.py", line 413, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/management/base.py", line 459, in execute
    output = self.handle(*args, **options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/netbox/extras/management/commands/reindex.py", line 95, in handle
    i = search_backend.cache(model.objects.iterator(), remove_existing=False)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/netbox/netbox/search/backends.py", line 197, in cache
    for instance in instances:
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/models/query.py", line 518, in _iterator
    yield from iterable
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/models/query.py", line 91, in __iter__
    results = compiler.execute_sql(
              ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1562, in execute_sql
    cursor.execute(sql, params)
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 79, in execute
    return self._execute_with_wrappers(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers
    return executor(sql, params, many, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 100, in _execute
    with self.db.wrap_database_errors:
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 105, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/psycopg/server_cursor.py", line 294, in execute
    raise ex.with_traceback(None)
django.db.utils.ProgrammingError: relation "netbox_topology_views_coordinategroup" does not exist
LINE 1: ...opology_views_coordinategroup"."description" FROM "netbox_to...
                                                             ^
root@netbox-local:/opt/netbox#

mattieserver · 2024-07-18T14:44:59Z

mattieserver
Jul 18, 2024
Maintainer

So you ran the command ./upgrade.sh?
And that worked, or does ./upgrade.sh also cause issues?

0 replies

deanfourie1 · 2024-07-18T20:37:56Z

deanfourie1
Jul 18, 2024
Author

If I dont have the plugin configured in local_requirements and configuration.py, the ./upgrade.sh completes without issues,

When I add the plugin back into my local_requirements and confguration.py, then the ./upgrade.sh fails with the above.

0 replies

deanfourie1 · 2024-07-21T20:40:27Z

deanfourie1
Jul 21, 2024
Author

Anyone>

0 replies

deanfourie1 · 2024-07-23T12:09:37Z

deanfourie1
Jul 23, 2024
Author

Any idea why I cannot install this plugin?

0 replies

deanfourie1 · 2024-08-03T09:02:22Z

deanfourie1
Aug 3, 2024
Author

Is there a way to recreate the tables?

0 replies

dreng · 2024-08-08T19:32:06Z

dreng
Aug 8, 2024
Collaborator

Is there a way to recreate the tables?

While you're talking about "recreating" tables, I don't see a proof that any tables have been created at all. Your log says:

Operations to perform:
Apply all migrations: netbox_topology_views
Running migrations:
No migrations to apply.

That can't be the whole truth. You can show all applied migrations in your venv with

python3 /opt/netbox/netbox/manage.py showmigrations

Have any migrations for netbox_topology_views been applied before?

In addition you can have a look at dbshell and check the tables:

Start dbshell
python3 /opt/netbox/netbox/manage.py dbshell
List tables
\dt
You may want to filter the list by tables starting with an "n"
\dt n*

It is possible to revert migrations (if you'd like to re-apply them), but to be honest, I've never done that. So be careful if you do that. Make a backup or snapshot first. This is what I have found: https://stackoverflow.com/questions/32123477/how-to-revert-the-last-migration

I'm converting this thread into a discussion, as it doesn't seem to be bug but an individual problem.

0 replies

PDS-GH · 2024-08-09T19:28:48Z

PDS-GH
Aug 9, 2024

A fresh install of the plugin on Netbox 4.0.8 fails in the same way.

# python3 manage.py migrate netbox_topology_views

Operations to perform:
  Apply all migrations: netbox_topology_views
Running migrations:
  Applying netbox_topology_views.0007_individualoptions_group_locations_and_more...Traceback (most recent call last):
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 103, in _execute
    return self.cursor.execute(sql)
  File "/opt/netbox/venv/lib/python3.10/site-packages/psycopg/cursor.py", line 97, in execute
    raise ex.with_traceback(None)
psycopg.errors.UndefinedTable: relation "netbox_topology_views_individualoptions" does not exist

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/netbox-4.0.8/netbox/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/management/base.py", line 413, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/management/base.py", line 459, in execute
    output = self.handle(*args, **options)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/management/base.py", line 107, in wrapper
    res = handle_func(*args, **kwargs)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/management/commands/migrate.py", line 356, in handle
    post_migrate_state = executor.migrate(
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/migrations/executor.py", line 135, in migrate
    state = self._migrate_all_forwards(
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/migrations/executor.py", line 167, in _migrate_all_forwards
    state = self.apply_migration(
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/migrations/executor.py", line 252, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/migrations/migration.py", line 132, in apply
    operation.database_forwards(
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/migrations/operations/fields.py", line 108, in database_forwards
    schema_editor.add_field(
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/base/schema.py", line 756, in add_field
    self.execute(sql, params or None)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/postgresql/schema.py", line 48, in execute
    return super().execute(sql, None)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/base/schema.py", line 202, in execute
    cursor.execute(sql, params)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 79, in execute
    return self._execute_with_wrappers(
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 100, in _execute
    with self.db.wrap_database_errors:
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 103, in _execute
    return self.cursor.execute(sql)
  File "/opt/netbox/venv/lib/python3.10/site-packages/psycopg/cursor.py", line 97, in execute
    raise ex.with_traceback(None)
django.db.utils.ProgrammingError: relation "netbox_topology_views_individualoptions" does not exist

# python3 manage.py showmigrations

[...]
netbox_topology_views
 [X] 0001_initial
 [X] 0002_individualoptions
 [X] 0003_individualoptions_show_neighbors
 [X] 0004_coordinategroup_coordinate
 [X] 0005_individualoptions_save_coords
 [X] 0006_powerpanelcoordinate_powerfeedcoordinate_and_more
 [ ] 0007_individualoptions_group_locations_and_more
[...]

# python3 manage.py dbshell

psql (14.12 (Ubuntu 14.12-0ubuntu0.22.04.1))
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, bits: 256, compression: off)
Type "help" for help.

netbox=> \dt n*

Did not find any relation named "n*".

It looks like there is no tables created for the plugin

0 replies

dreng · 2024-08-14T12:42:40Z

dreng
Aug 14, 2024
Collaborator

Migration 007 complains that the table "netbox_topology_views_individualoptions" doesn't exist.

If you have a look at your migrations you can see that migration 001 to 006 have been applied before (marked with an [x]). Nevertheless, the tables that should have been created for 001 to 006 are missing.

This leads me to the assumption that you have installed a lower version version of NTV before and restored an older database version after that. This would explain why the tables are gone.

It is possible to revert migrations (if you'd like to re-apply them), but to be honest, I've never done that. So be careful if you do that. Make a backup or snapshot first. This is what I have found: https://stackoverflow.com/questions/32123477/how-to-revert-the-last-migration

0 replies

deanfourie1 · 2024-08-14T22:37:21Z

deanfourie1
Aug 14, 2024
Author

I got this working by manually creating the tables.

0 replies

PDS-GH · 2024-08-15T00:06:36Z

PDS-GH
Aug 15, 2024

Finally got it working by following the removal procedure here : https://netboxlabs.com/docs/netbox/en/stable/plugins/removal/
The important part is removing the Django Migration Records (tables for the plugin in DB were already missing).
Then I reinstalled the plugin.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Failing to install on 4.0 #544

{{title}}

Replies: 10 comments

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

Select a reply

Failing to install on 4.0 #544

deanfourie1 Jul 18, 2024

NetBox version

Topology Views version

Steps to Reproduce

Expected Behavior

Observed Behavior

Replies: 10 comments

mattieserver Jul 18, 2024 Maintainer

deanfourie1 Jul 18, 2024 Author

deanfourie1 Jul 21, 2024 Author

deanfourie1 Jul 23, 2024 Author

deanfourie1 Aug 3, 2024 Author

dreng Aug 8, 2024 Collaborator

PDS-GH Aug 9, 2024

dreng Aug 14, 2024 Collaborator

deanfourie1 Aug 14, 2024 Author

PDS-GH Aug 15, 2024

deanfourie1
Jul 18, 2024

mattieserver
Jul 18, 2024
Maintainer

deanfourie1
Jul 18, 2024
Author

deanfourie1
Jul 21, 2024
Author

deanfourie1
Jul 23, 2024
Author

deanfourie1
Aug 3, 2024
Author

dreng
Aug 8, 2024
Collaborator

PDS-GH
Aug 9, 2024

dreng
Aug 14, 2024
Collaborator

deanfourie1
Aug 14, 2024
Author

PDS-GH
Aug 15, 2024