Upgrading 0.5.12 to 0.6.4 failed, and directly using 0.6.4 to start the program also failed. #1274

lsg328 · 2024-01-02T04:13:35Z

Describe the bug
When starting Langflow with PostgreSQL, an error occurs when upgrading from 0.5.12 to 0.6.4. Even when attempting to start directly with version 0.6.4 and a new database, errors persist, specifically between versions 0.6.0 - 0.6.4.

Version

Version 0.6.0 - 0.6.4.

Log
upgrading from 0.5.12 to 0.6.4

[2024-01-02 11:58:28 +0800] [10953] [ERROR] Traceback (most recent call last):
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/langflow/services/database/service.py", line 149, in run_migrations
    command.check(alembic_cfg)
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/alembic/command.py", line 290, in check
    script_directory.run_env()
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/alembic/script/base.py", line 583, in run_env
    util.load_python_file(self.dir, "env.py")
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/alembic/util/pyfiles.py", line 95, in load_python_file
    module = load_module_py(module_id, path)
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/alembic/util/pyfiles.py", line 113, in load_module_py
    spec.loader.exec_module(module)  # type: ignore
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/langflow/alembic/env.py", line 81, in <module>
    run_migrations_online()
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/langflow/alembic/env.py", line 75, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/alembic/runtime/environment.py", line 948, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/alembic/runtime/migration.py", line 615, in run_migrations
    for step in self._migrations_fn(heads, self):
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/alembic/command.py", line 279, in retrieve_migrations
    revision_context.run_autogenerate(rev, context)
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/alembic/autogenerate/api.py", line 570, in run_autogenerate
    self._run_environment(rev, migration_context, True)
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/alembic/autogenerate/api.py", line 591, in _run_environment
    raise util.CommandError("Target database is not up to date.")
alembic.util.exc.CommandError: Target database is not up to date.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1969, in _exec_single_context
    self.dialect.do_execute(
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 922, in do_execute
    cursor.execute(statement, parameters)
psycopg2.errors.InFailedSqlTransaction: current transaction is aborted, commands ignored until end of transaction block


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/starlette/routing.py", line 705, in lifespan
    async with self.lifespan_context(app) as maybe_state:
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/contextlib.py", line 199, in __aenter__
    return await anext(self.gen)
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/langflow/main.py", line 19, in lifespan
    initialize_services()
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/langflow/services/utils.py", line 206, in initialize_services
    raise exc
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/langflow/services/utils.py", line 203, in initialize_services
    initialize_database(fix_migration=fix_migration)
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/langflow/services/database/utils.py", line 52, in initialize_database
    raise exc
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/langflow/services/database/utils.py", line 32, in initialize_database
    database_service.run_migrations(fix=fix_migration)
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/langflow/services/database/service.py", line 152, in run_migrations
    command.upgrade(alembic_cfg, "head")
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/alembic/command.py", line 403, in upgrade
    script.run_env()
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/alembic/script/base.py", line 583, in run_env
    util.load_python_file(self.dir, "env.py")
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/alembic/util/pyfiles.py", line 95, in load_python_file
    module = load_module_py(module_id, path)
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/alembic/util/pyfiles.py", line 113, in load_module_py
    spec.loader.exec_module(module)  # type: ignore
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/langflow/alembic/env.py", line 81, in <module>
    run_migrations_online()
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/langflow/alembic/env.py", line 75, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/alembic/runtime/environment.py", line 948, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/alembic/runtime/migration.py", line 627, in run_migrations
    step.migration_fn(**kw)
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/langflow/alembic/versions/7d2162acc8b2_adds_updated_at_and_folder_cols.py", line 32, in upgrade
    with op.batch_alter_table('apikey', schema=None) as batch_op:
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/contextlib.py", line 142, in __exit__
    next(self.gen)
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/alembic/operations/base.py", line 398, in batch_alter_table
    impl.flush()
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/alembic/operations/batch.py", line 116, in flush
    fn(*arg, **kw)
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/alembic/ddl/postgresql.py", line 187, in alter_column
    super().alter_column(
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/alembic/ddl/impl.py", line 241, in alter_column
    self._exec(
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/alembic/ddl/impl.py", line 207, in _exec
    return conn.execute(construct, multiparams)
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1416, in execute
    return meth(
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/sqlalchemy/sql/ddl.py", line 181, in _execute_on_connection
    return connection._execute_ddl(
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1528, in _execute_ddl
    ret = self._execute_context(
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1848, in _execute_context
    return self._exec_single_context(
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1988, in _exec_single_context
    self._handle_dbapi_exception(
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2344, in _handle_dbapi_exception
    raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1969, in _exec_single_context
    self.dialect.do_execute(
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 922, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.InternalError: (psycopg2.errors.InFailedSqlTransaction) current transaction is aborted, commands ignored until end of transaction block

start directly with version 0.6.4 and a new database

                             RuntimeError: Something went wrong running
                             migrations. Please, run `langflow migration --fix`
[2024-01-02 12:11:46 +0800] [11793] [ERROR] Traceback (most recent call last):
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/langflow/services/database/service.py", line 156, in run_migrations
    command.check(alembic_cfg)
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/alembic/command.py", line 300, in check
    raise util.AutogenerateDiffsDetected(
alembic.util.exc.AutogenerateDiffsDetected: New upgrade operations detected: [('add_constraint', UniqueConstraint(Column('id', NullType(), table=<apikey>))), ('add_constraint', UniqueConstraint(Column('id', NullType(), table=<flow>))), ('add_constraint', UniqueConstraint(Column('id', NullType(), table=<user>)))]

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/starlette/routing.py", line 705, in lifespan
    async with self.lifespan_context(app) as maybe_state:
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/contextlib.py", line 199, in __aenter__
    return await anext(self.gen)
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/langflow/main.py", line 19, in lifespan
    initialize_services()
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/langflow/services/utils.py", line 206, in initialize_services
    raise exc
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/langflow/services/utils.py", line 203, in initialize_services
    initialize_database(fix_migration=fix_migration)
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/langflow/services/database/utils.py", line 52, in initialize_database
    raise exc
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/langflow/services/database/utils.py", line 32, in initialize_database
    database_service.run_migrations(fix=fix_migration)
  File "/usr/lib/anaconda3/envs/langflow-0.6.4/lib/python3.10/site-packages/langflow/services/database/service.py", line 160, in run_migrations
    raise RuntimeError(
RuntimeError: Something went wrong running migrations. Please, run `langflow migration --fix

[SQL: ALTER TABLE apikey ALTER COLUMN name SET NOT NULL]
(Background on this error at: https://sqlalche.me/e/20/2j85)

Screenshots
upgrading from 0.5.12 to 0.6.4

start directly with version 0.6.4 and a new database

The text was updated successfully, but these errors were encountered:

dosubot · 2024-01-02T04:14:05Z

🤖

Hi @lsg328! I'm Dosu, a friendly bot here to lend a hand. While you're waiting for a human maintainer, I'll do my best to help you solve bugs, answer queries, and guide you on your journey as a contributor. Let's iron out these wrinkles together!

Based on the information you've provided, it seems like the issue you're experiencing is related to the database migration scripts between versions 0.5.12 and 0.6.4.

The error "Target database is not up to date" could be due to several changes made in the migration scripts. For instance, the script tries to drop two indexes and two tables, alter a column to be non-nullable, add two new columns, and recreate the dropped tables and indexes. If any of these operations fail, it could result in the error you're seeing. You can find more details about these changes in the migration script.

The SQL command "ALTER TABLE apikey ALTER COLUMN name SET NOT NULL" seems to be causing an issue. This command is trying to alter the 'name' column of the 'apikey' table to be not nullable. If there are any rows in the 'apikey' table where the 'name' column is null, this operation will fail because it would violate the new constraint of the 'name' column being not nullable.

To fix this issue, you could modify the downgrade() function to first update any null values in the 'name' column to a non-null value before altering the column to be not nullable. Here's an example of how you could do this:

def downgrade() -> None:
    # ### commands auto generated by Alembic - please adjust! ###
    try:
        op.execute("UPDATE apikey SET name = 'default' WHERE name IS NULL")
        with op.batch_alter_table('apikey', schema=None) as batch_op:
            batch_op.alter_column('name',
                existing_type=sa.VARCHAR(),
                nullable=False)
    except Exception as e:
        pass
    # ### end Alembic commands ###

In this example, any null values in the 'name' column are updated to 'default' before the column is altered to be not nullable. Please adjust the default value ('default' in this case) to fit your application's needs.

There are also similar solved issues in the langflow repository that you might find helpful:

Alembic DB migration error in Langflow v0.6.0 with PostgreSQL which was last updated on December 13, 2023
The migration is failed when the database is postgresql which was last updated on October 10, 2023

I hope this helps! If you need further assistance, please provide more specific information about the database state and the exact error message.

Sources

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

lsg328 · 2024-01-02T04:30:29Z

Manually executing the SQL statement "ALTER TABLE apikey ALTER COLUMN name SET NOT NULL" can be executed normally. And if you use the new database directly (no tables exist), you will also get an error.

lsg328 · 2024-02-27T10:29:00Z

@ogabrielluiz Versions after langflow0.5.12 will have this problem when using postgreSQL, whether it is an upgrade or a new environment

dosubot · 2024-02-27T10:34:40Z

🤖

Based on the information you've provided and the context from the langflow repository, it seems like you're encountering a series of errors related to database migrations when upgrading Langflow from version 0.5.12 to 0.6.4 or starting directly with version 0.6.4 with a new database.

The errors you're encountering could be due to several reasons:

CommandError: This error usually occurs when there's an issue with the SQL command being executed. In your case, it could be due to the batch_alter_table commands in the upgrade() and downgrade() functions. If the 'flow' table doesn't exist or if the columns being added or dropped don't exist, it could lead to a CommandError.
psycopg2.errors.InFailedSqlTransaction: This error is raised when a SQL command cannot be executed because a transaction block is in an aborted state (due to a previous command that failed). In your case, it could be due to a previous failed command in the upgrade() function.
RuntimeError: This is a generic error that can be caused by numerous issues. Without more specific information, it's hard to pinpoint the exact cause. However, it could be related to the other errors you're encountering.
alembic.util.exc.AutogenerateDiffsDetected: This error is raised when Alembic detects differences between the current database schema and the schema defined in your models. This could be due to changes in your models that haven't been reflected in a migration script.

To resolve these issues, you could try the following:

Ensure that the 'flow' table and the columns being added or dropped exist in your database.
Check the order of your migration scripts. If a script is trying to modify a table or column that hasn't been created yet, it could lead to errors.
If you've made changes to your models, make sure to generate a new migration script to reflect these changes.
If a transaction block is in an aborted state, you might need to rollback the transaction before you can execute new commands.

Please note that without more specific information about your database schema and the exact errors you're encountering, it's hard to provide a more precise solution.

Regarding the changes made in the migration scripts between versions 0.6.0 and 0.6.4 of Langflow, you can refer to the migration script for more details.

Lastly, there are similar issues that have been solved in the langflow repository. You might find the discussions and solutions in these issues helpful:

I hope this information helps you in resolving the issues you're facing. If you need further assistance, please provide more details about your database schema and the exact errors you're encountering.

Sources

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

lsg328 · 2024-02-29T04:22:22Z

Problems still exist with the latest version

dosubot · 2024-02-29T04:24:31Z

🤖

It seems like this issue requires a deeper look, @ogabrielluiz could you please lend your expertise?

ogabrielluiz · 2024-02-29T11:05:25Z

We will work on that ASAP. Probably today or tomorrow.
Sorry for the delayed response. We've been hard at work on the new release

ogabrielluiz · 2024-02-29T19:04:35Z

@lsg328 Could you test version 0.6.9 if it is working on your end?

dosubot bot added the bug Something isn't working label Jan 2, 2024

This was referenced Feb 29, 2024

Refactor code and fix database issues #1484

Merged

Refactor error page layout and fixes migration scripts #1485

Merged

ogabrielluiz linked a pull request Feb 29, 2024 that will close this issue

Refactor error page layout and fixes migration scripts #1485

Merged

ogabrielluiz closed this as completed in 4bec4de Mar 4, 2024

dosubot bot mentioned this issue Apr 4, 2024

Postgres migration error #1607

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrading 0.5.12 to 0.6.4 failed, and directly using 0.6.4 to start the program also failed. #1274

Upgrading 0.5.12 to 0.6.4 failed, and directly using 0.6.4 to start the program also failed. #1274

lsg328 commented Jan 2, 2024 •

edited

Loading

dosubot bot commented Jan 2, 2024 •

edited

Loading

lsg328 commented Jan 2, 2024

lsg328 commented Feb 27, 2024

dosubot bot commented Feb 27, 2024

About Dosu

lsg328 commented Feb 29, 2024

dosubot bot commented Feb 29, 2024

ogabrielluiz commented Feb 29, 2024

ogabrielluiz commented Feb 29, 2024

Upgrading 0.5.12 to 0.6.4 failed, and directly using 0.6.4 to start the program also failed. #1274

Upgrading 0.5.12 to 0.6.4 failed, and directly using 0.6.4 to start the program also failed. #1274

Comments

lsg328 commented Jan 2, 2024 • edited Loading

dosubot bot commented Jan 2, 2024 • edited Loading

Sources

lsg328 commented Jan 2, 2024

lsg328 commented Feb 27, 2024

dosubot bot commented Feb 27, 2024

Sources

About Dosu

lsg328 commented Feb 29, 2024

dosubot bot commented Feb 29, 2024

ogabrielluiz commented Feb 29, 2024

ogabrielluiz commented Feb 29, 2024

lsg328 commented Jan 2, 2024 •

edited

Loading

dosubot bot commented Jan 2, 2024 •

edited

Loading