I think have some serious issues: upgrade 3.0.2 to 3.0.3

[garethjax]

How Shlink is set up

• Shlink Version: 3.0.2
• PHP Version:PHP 8.1.13 (cli) (built: Nov 22 2022 14:42:07) (NTS gcc x86_64)
• How do you serve Shlink: Self-hosted Apache
• Database engine used: SQLite

Summary

i've been trying to download and use several versions of shlink and i cannot even upgrade 3.0.2 to 3.0.3., but i'm always getting a doctrine error like this. (this is just an extract).

I'm supposing that the hosting i'm using (opalstack) has several limitations which i'm probably hitting. I should probably try to move the shortener to another dedicated server that i own. I may have to migrate it entirely.

Error Output:
ERR ================
ERR
ERR In ExceptionConverter.php line 71:
ERR
ERR
ERR [Doctrine\DBAL\Exception\SyntaxErrorException (1)]
ERR
ERR An exception occurred while executing a query: SQLSTATE[HY000]: General e
ERR rr
ERR or: 1 near "(": syntax error
ERR
ERR
ERR
ERR
ERR Exception trace:
ERR at /home/tracciaclick/shlink303/vendor/doctrine/dbal/src/Driver/API/SQLit
ERR e/ExceptionConverter.php:71

[garethjax]

Wait wait wait, maybe i'm a dumb idiot...

edit: the errors in the links mentioned before (now removed from the message) was caused by an error in my cli commands.

The migration error remains.

This is the PHP version on Opalstack

PHP 8.0.23 (cli) (built: Aug 30 2022 12:16:34) ( NTS gcc x86_64 )
Copyright (c) The PHP Group
Zend Engine v4.0.23, Copyright (c) Zend Technologies
with Zend OPcache v8.0.23, Copyright (c), by Zend Technologies

test install and migration on another server

• i've created a new configuration on a server i own
• i've downloaded shlink 3.0.2 for php 8.0 (this server doesn't have 8.1 and 8.2 yet)
• i've zipped the old folder and expanded it in a private folder in the new server
• launched the migration using php 8.0
• the migration seems to work!

this server has a much older version of php:

PHP 8.0.13 (cli) (built: Nov 26 2021 07:45:11) ( NTS )
Copyright (c) The PHP Group
Zend Engine v4.0.13, Copyright (c) Zend Technologies

/opt/php-8.0/bin/php' 'bin/cli' 'db:migrate'
OUT Migrating database...
OUT
ERR RUN '/opt/php-8.0/bin/php' 'vendor/doctrine/migrations/bin/doctrine-migrations.php' 'migrations:migrate' '--no-interaction'
ERR OUT
ERR OUT
ERR RES Command ran successfully
ERR
OUT
OUT [OK] Database properly migrated!
OUT
OUT
RES Command ran successfully
Success!
Generating proxies... [Running "/opt/php-8.0/bin/php vendor/doctrine/orm/bin/doctrine.php orm:generate-proxies"] RUN '/opt/php-8.0/bin/php' 'vendor/doctrine/orm/bin/doctrine.php' 'orm:generate-proxies'
OUT
OUT Processing entity "Shlinkio\Shlink\Core\Entity\ShortUrl"
OUT Processing entity "Shlinkio\Shlink\Core\Entity\VisitLocation"
OUT Processing entity "Shlinkio\Shlink\Core\Entity\Domain"
OUT Processing entity "Shlinkio\Shlink\Core\Entity\Tag"
OUT Processing entity "Shlinkio\Shlink\Core\Entity\Visit"
OUT Processing entity "Shlinkio\Shlink\Rest\Entity\ApiKeyRole"
OUT Processing entity "Shlinkio\Shlink\Rest\Entity\ApiKey"
OUT
OUT Proxy classes generated to "/web/data/proxies"
OUT
RES Command ran successfully
Success!
Clearing entities cache... [Running "/opt/php-8.0/bin/php vendor/doctrine/orm/bin/doctrine.php orm:clear-cache:metadata"] RUN '/opt/php-8.0/bin/php' 'vendor/doctrine/orm/bin/doctrine.php' 'orm:clear-cache:metadata'
OUT
OUT // Clearing all Metadata cache entries
OUT
OUT [OK] Successfully deleted cache entries.
OUT
OUT
RES Command ran successfully
Success!

[acelaya]

It's hard to tell. I have never experienced that myself, but it's definitely not good. SQLite is not designed for that kind of usage.

[garethjax]

The resolution won't be fast :) the server doesn't support php 8.2 (i have to migrate from debian 9 to 11).

[acelaya]

I think it's more critical to migrate to a proper database engine.

[garethjax]

how should i migrate, using the same hosting space?
maybe i should create a new install in a separate folder, migrate the previous data and the switch the two folders?

[acelaya]

Correct

[garethjax]

Update:

• the clean install worked this time (version 8.1)
• i've connected a brand new mariadb database
• when i try to import the data, including the visits, but without the orphaned clicks, the script exits with an error, after having processed about 10 links (of 116)

PHP Fatal error: Allowed memory size of 524288000 bytes exhausted (tried to allocate 10485760 bytes) in /home/tracciaclick/shlink352-81/vendor/doctrine/orm/lib/Doctrine/ORM/UnitOfWork.php on line 1381

Does the migration entirely happens in memory before writing to the Database?

[garethjax]

Check who's returning those 401 errors. Are those coming from Shlink or the web server? Have you checked Shlink logs (inside data/log folder) and the web server logs (location depends on OS).

Also, from the network tab, check what's the actual response body for those requests returning a 401 error.

Regarding this part, i still have to do the test, let me review the procedure:

• i need to change the dns of the domain (because they are two different hostings and they handle SSL differently, one is managed by the provider and the new one is using cloudflare)
• when the dns configuration has been updated, i'll try to connect again to the shlink instance using the web app.
• the old configuration should stop working and the new one should work

some doubts

• do the react app uses a server component in order to connect to the instance? There could be a delay in the name resolution?
• is it possible to "use" a local domain name resolution managed using the /etc/hosts config on my device?
• i've already seen that it's not possible to connect over "http:" protocol because the app deny those connections, is it possible to bypass it for testing ?

thanks for your time!

[acelaya]

I think you are facing two unrelated issues:

• The error you see from the CLI is probably caused by an incorrect installation process, for example if it was not run from the project's root folder.
  Another reason could be that there are missing files, but I supposed you did not go deleting random files 😅
• The 401 error you see from shlink-web-client is probably a DNS misconfiguration.

Second issue (401s)

If you visit https://oraoraora.xyz/rest/health, you will see the version displayed is 3.0.2, which means it's still your old instance. Since you are passing the API key from the new one, it is rejecting the requests.

In any case, once you solve that, you will start facing the first issue also from a web context.

First issue (ServiceManager exception)

What this error means is that Shlink cannot find the file module/Core/config/entities-mappings/Shlinkio.Shlink.Core.ShortUrl.Entity.ShortUrl.php, which means it is looking for it in the wrong place.

As I mentioned above, one possible reason is that the installation was not run from the project's root folder, which means some config cache could have been generated containing incorrect paths.

Check if there's a data/cache/app_config.php file and delete it. If it doesn't, make sure the file I referenced above does in fact exist.

Also, please confirm you did not change the directory structure in any way.

Other questions

Now answering some of your other questions:

Regarding this part, i still have to do the test, let me review the procedure

• ...

The steps you describe above are correct. This is what is causing the second issue.

do the react app uses a server component in order to connect to the instance? There could be a delay in the name resolution?

Nop, it runs 100% in your browser.

is it possible to "use" a local domain name resolution managed using the /etc/hosts config on my device?

Yes, it's possible. The Shlink servers are called from the browser, which is in your local machine, so it will resolve names as defined in /etc/hosts.

There's a consideration regarding HTTPS, which I'll answer below.

i've already seen that it's not possible to connect over "http:" protocol because the app deny those connections, is it possible to bypass it for testing ?

If shlink-web-client runs on HTTPS, the browser will not allow connections to non-HTTPS endpoints, so this is not possible using https://app.shlink.io

The only exception is if the server is localhost. I do this all the time when developing Shlink, running it locally on localhost:<port> and using https://app.shlink.io as a convenient way to consume it.

If you want to use arbitrary names mapped in /etc/hosts, then I would recommend running shlink-web-client locally too: Try the second or third option from https://shlink.io/documentation/shlink-web-client/installation/

[garethjax]

What this error means is that Shlink cannot find the file module/Core/config/entities-mappings/Shlinkio.Shlink.Core.ShortUrl.Entity.ShortUrl.php, which means it is looking for it in the wrong place.

As I mentioned above, one possible reason is that the installation was not run from the project's root folder, which means some config cache could have been generated containing incorrect paths.

Check if there's a data/cache/app_config.php file and delete it. If it doesn't, make sure the file I referenced above does in fact exist.

Also, please confirm you did not change the directory structure in any way.

regarding the installation

i have a history of the commands from shell

  114  wget https://github.com/shlinkio/shlink/releases/download/v3.5.4/shlink3.5.4_php8.2_dist.zip
  115  ls -la
  116  unzip shlink3.5.4_php8.2_dist.zip

  130  cd shlink3.5.4_php8.2_dist
  131  ls -la
  132   mv * /web
  
139  cd /web (it's the public folder for the  website)
  141  php8.2 vendor/bin/shlink-installer update

about the rest

• the file app_config.php is there, i will delete it
• i haven't changed anything, i've just done the migration of the url/clicks from the original server (successfully).

[garethjax]

If shlink-web-client runs on HTTPS, the browser will not allow connections to non-HTTPS endpoints, so this is not possible using https://app.shlink.io

The only exception is if the server is localhost. I do this all the time when developing Shlink, running it locally on localhost:<port> and using https://app.shlink.io as a convenient way to consume it.

If you want to use arbitrary names mapped in /etc/hosts, then I would recommend running shlink-web-client locally too: Try the second or third option from https://shlink.io/documentation/shlink-web-client/installation/

the local install worked!

I'm starting to think a theory: the issue is about the SSL protocol (as you said).

• the old configuration is mapped to the (Opalstack) server: it is managing the SSL configuration "directly", since they ask a certificate from let's encrypt and load it in the hosting space.
• the new configuration is passing through cloudflare. The certificate is handled by cloudflare in the "flexible" configuration, so they "probably" pass the request to the API in http... and this is causing the errors.

Let me know what you think, in the mean time I'll ask my system administrator how to configure a let's encrypt certificate to add a proper SSL communication.

[acelaya]

  114  wget https://github.com/shlinkio/shlink/releases/download/v3.5.4/shlink3.5.4_php8.2_dist.zip
    115  ls -la
    116  unzip shlink3.5.4_php8.2_dist.zip
  
    130  cd shlink3.5.4_php8.2_dist
    131  ls -la
    132   mv * /web
    
  139  cd /web (it's the public folder for the  website)
    141  php8.2 vendor/bin/shlink-installer update

I see two potential problems there.

• You are going inside the folder and then moving its contents with mv * <destination>, instead of moving the whole folder. That could leave some files uncopied.
• You are copying the whole content to the document root, while Shlink is not designed to be served that way.

Instead of doing that, just unzip the folder, move the whole folder wherever you want to have it, run the update command, and configure the server to point the document root to /path/to/shlink/public. It's important that only the public folder is set as document root (when using apache or nginx, which I assume is the case).

> wget https://github.com/shlinkio/shlink/releases/download/v3.5.4/shlink3.5.4_php8.2_dist.zip
> unzip shlink3.5.4_php8.2_dist.zip
> mv hlink3.5.4_php8.2_dist <wherever>
> cd <wherever>
>  php8.2 vendor/bin/shlink-installer update

The vhost configuration depends on your web server, but you can check these https://shlink.io/documentation/supported-runtimes/classic-web-server/

[garethjax]

Hello, i'm going to close the discussion with some feedbacks for other people. :)

I was able to migrate and activate the new installation: some feedbacks

• if you have a multi-domain configuration, remember to set the folder "public" as default "path" for the web configuration.
  -- for example if you install into /www/domainname/ the zip will contain a subfolder /public
  -- the default path for the home should be /www/domainname/public
  -- it's made for safety reasons, to avoid access to the code from malevolent external scripts.
• if you use cloudflare to implement a SSL connection, be aware that the "flexible mode" is not enough, because the connection from cloudflare to your server is still using the http protocol and the web client will not work if you . You need to have a proper "strict" connection (for example by adding a let's encrypt certificate).
• If you need to do a migration, you can test to the new installation by
  -- mapping the domain locally changing your HOSTS file
  -- installing a copy of the web client on your computer and running it
  -- you need to configure the connection string using Http://yourdomainname
  -- Since you're using the http connection from your computer to the new install, the connection will be successful.
• the testing is vital, because you will probably need to change the DNS settings in order to migrate the domain from the old server to the new server. Switching the DNS configuration is not problematic, but the migration of the new information may take several hours before it's replicated worldwide, so you risk a loss of tracking if the new configuration has issues.
• The migration of data from one server to another was done without trouble, but the new server has LOTS of ram (32 GB), your results may be different if you use a small shared hosting. I didn't follow the complete process but it probably took around 15 minutes, for 1.000.000 visits and 118 short urls.

remaining issues for my install

I'm still getting those messages from cli, even if simply try to launch it using

• php8.2 bin/cli
• (if i try to ask for other features i receive similar messages)

In ServiceManager.php line 627:

  Service with name "Shlinkio\Shlink\Core\ShortUrl\Repository\ShortUrlListRepository" could not be created. Reason: No mapping file found named 'Shlinkio.Shlink.C
  ore.ShortUrl.Entity.ShortUrl.php' for class 'Shlinkio\Shlink\Core\ShortUrl\Entity\ShortUrl'.

In MappingException.php line 60:

  No mapping file found named 'Shlinkio.Shlink.Core.ShortUrl.Entity.ShortUrl.php' for class 'Shlinkio\Shlink\Core\ShortUrl\Entity\ShortUrl'

and weirdest yet:

• i've downloaded a copy of the script in a subfolder called _test
• expanded the zip
• from there the php8.2 bin/cli command works! (of course there are no configurations)

I've copied every folder from _test to the main install (beside the "data" folder), i'm still getting the same error.

...and it's even more weird...

i usually run all of those operations from a customized ssh connection, which is in "jailkit": it has permission to "act" only in the context of the domain.

Which is fine if i have to share the connection with a collaborator: they will be able to access only the files of the domain and nothing else.

Hear this: if i log to the server as ROOT, navigate the domain folder (which would be "/var/www/domainname/web") and try to launch the same php command... it works!

I think it's a nasty bug hidden somewhere: the jailkit "aliased path" is not parsed correctly from doctrine and probably prints this kind of error because it doesn't know how to handle /web . And it works in a subfolder of /web/_test because... reasons?

Anyway i think it's a only a problem limited to my configuration, i've just reported it for a future problem. It's probably easier to run shlink in a docker container :)

[acelaya]

Thanks for the detailed explanation! I'm sure it will be useful for other users.

I took the liberty to mark this as the correct answer, and re-open the discussion (I try to keep all discussions open by default).

As per some of your points:

if you have a multi-domain configuration, remember to set the folder "public" as default "path" for the web configuration

This is not limited to multi-domain. The public folder should always be set as the service's document root when not using RoadRunner or openswoole to serve Shlink.

if you use cloudflare to implement a SSL connection, be aware that the "flexible mode" is not enough, because the connection from cloudflare to your server is still using the http protocol and the web client will not work if you

I'm not completely sure how "flexible mode" works in Cloudflare, but as long as Cloudflare works as reverse proxy cache, it doesn't matter how Cloudflare and Shlink communicate. From the web client point of view, it's HTTPS, so I think the problem here was different.

With that I don't mean to say it's not a good idea that Shlink and Cloudflare communicate via HTTPS, just that this should not cause problems for shlink-web-client.

I'm still getting those messages from cli
No mapping file found named 'Shlinkio.Shlink.Core.ShortUrl.Entity.ShortUrl.php' for class 'Shlinkio\Shlink\Core\ShortUrl\Entity\ShortUrl'
[...]
if i log to the server as ROOT, navigate the domain folder (which would be "/var/www/domainname/web") and try to launch the same php command... it works!

This is actually an important point.

It's true that Shlink has usually focused on supporting docker better than things like shared hostings, because there are other solutions covering that.

However, the VPS use case is something I want to keep supporting, and the usage of jailkit is relatively common. It might affect some aspects of Shlink in which access to certain directories is important, so probably something worth investigating a bit further.

But yes, I wouldn't be surprised this is what is causing problems for you.

[garethjax]

No problem in reopening the discussion, i've forgot that it could be marked as "solved" !

I'm not completely sure how "flexible mode" works in Cloudflare, but as long as Cloudflare works as reverse proxy cache, it doesn't matter how Cloudflare and Shlink communicate. From the web client point of view, it's HTTPS, so I think the problem here was different.

With that I don't mean to say it's not a good idea that Shlink and Cloudflare communicate via HTTPS, just that this should not cause problems for shlink-web-client.

It's a partial implementation of SSL, it uses a secured protocol from cloudflare to the outside world. Everything that's behind cloudflare is unencrypted.

https://developers.cloudflare.com/ssl/origin-configuration/ssl-modes/flexible/

it's a good compromise, because it's free and easy to implement even on a shared hosting (everything happens in the cloudflare structure).

I would NOT recommend it for a critical trasmission of information (eg: credit cards, personal informations), but if you want to reassure the visitors (and the pesky Google), it works pretty well in general uses.

From the webclient point of view, you probably switch from https to http and therefore you receive a 401 error message in the console. ( and i think i've seen it )