-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Instructions on how to migrate from Misskey YunoHost to Calckey YunoHost #37
Comments
Thanks for sharing the instructions, very handy. That is something we should implement fully on our side to automate it. To do it manually, from what I can understand after a quick look, it seems the first part of the script that applies patches can be done in Yunohost without any issue. The Calckey install part is a bit more tricky. My naive try would be:
This is a classical method for software that share the same kind of database (it will import it), assuming they are at least a bit compatible (= it can start and run migrations if needed). To be tested in a non critical situation (you may loose everything, except if your backup is good), it will surely fail somewhere. |
That seems like a reasonable approach. I will try it and tell you what happens. |
Okay. Well, I tried that. I copied the "db.sql" file and the "files" directory in my YunoHost Misskey backup into my YunoHost Calckey backup (that is, replaced the Calckey versions) and then restored. Intriguingly, the files directory from Calckey was not empty -- examining it, I find content from my contacts. I assume it must be getting federation messages based on the domain being the same as before? It comes up with "Internal Server Error" in my browser, even after several reloads. I restarted the "Calckey" service a couple of times to see if that would help. The log of (the 3rd?) restart looked like this: Jun 11 05:13:07 systemd[1]: Stopping Calckey: fork of Misskey... Now, I'm wondering if that "pnpm" command will eventually migrate the database or something? So I'm not sure if this is actually a failure, yet, or just that I haven't given it enough time. The database backup was about 980 MB and there's about 20GB of data in files. So it might take awhile to finish processing them if it needs to do that. (?) |
It's still running, but it produces a lot of "missing column" errors, like: Jun 11 06:44:33 npm[2689]: WARN 1 [queue inbox] failed(QueryFailedError: column Meta.disableRecommendedTimeline does not exist) id=283 attempts=4/8 age=11m activity=https://social.vivaldi.net/users/mykillenmy#delete |
Here is the limit of what I can help to debug, I have no idea where to start because of my lack of *key internal DB structure knowledge. |
Okay. Well, I thought there might be some auto-healing built in to recover from the missing columns issues, but it hasn't recovered after running overnight, so I'm guessing not. Seems like this method fails. I will have to take some time to decide whether it's worth pursuing a more complex solution, or just write off the loss of either my Misskey instance/accounts or the Calckey software. I'm not sure how attached I really am to either, so either or both are certainly options. It has been an interesting experiment. Thank you very much for your help! |
I'm not sure I wasn't clear, but I think this method might work, but with adjustments because I don't know which step to run at which point, in particular for the migration and patch stuff. |
Yeah. I understood. |
That means that migrations weren't run. Is there any way you can manually run |
Okay. I have repeated the process as before. Except, I then installed pnpm:
Moved to the Calckey directory:
And ran the proposed command:
It is not a happy camper, though. I'll paste the entire output here. Sorry for the length, but I'm not sure which bit is important! It looks like maybe there is something I have to clean up first?
|
Could you edit it to use "`" instead ? It would be a lot more readable Thank you for documenting this. |
I mean, this traceback is unfamiliar to me, but it looks to me like the relevant bit is probably:
Just to be clear about my skills: Kind of on the edge with PostgreSQL. I know RDBMS conceptually and I've used MySQL. I think I have the Adminer YunoHost app installed, so I probably could use that to look at the database, but I have no experience with it, yet. Totally out of my depth with npm, pnpm, Typsecript, modern Javascript libraries, Node.js (when I did Javascript many years ago, it was a little toy language you could use for stupid HTML stricks). I use Python for most of my coding needs, so it's a whole other environment. Once upon a time I wrote C, C++, and Fortran, but it's been decades. |
We will definitely need someone with some understanding of that stuff to investigate… |
Can I ask what version of Misskey you're migrating from? |
Also, after some fiddling, I found that the backup archive also contains the database name, username, and password -- all for "calckey". These would be the ones from the newly-created Calckey that I backed up to create the merged archive. But I can get into that with Adminer using these credentials, if it would be useful to get some table info? (But I don't know what to look for). Or, if it's simpler, I can probably do the same with psql on the command line. |
I'm assuming that the db.sql file just loads the database contents, so they will be in this 'calckey' database now, even though they came from the 'misskey' database before. I tried using the misskey credentials and database, but Adminer gives me an authentication error. I suspect that database no longer exists after uninstalling misskey, but I'm unsure how else to prove that. |
If you restored a calckey backup, yes it does.
Yes. Uninstall an app removes everything, including the database (ex: https://github.com/YunoHost-Apps/misskey_ynh/blob/master/scripts/remove#L60). |
Okay, one thing that looks a little off: I got into psql using the credentials for calckey (so I'm user 'calckey' in database 'calckey'), and I list the tables in the database:
The list shows all the tables are owned by
I was thinking that maybe I could just trying deleting the But it won't let me do that as Is that right? Seems like the point of having a Calckey database and username would be to interact as I'm not 100% sure I can log in as (I still have the merged backup, of course, so if I wreck it, I can start over). |
I need to check the owner on a normal installation, but that probably has to be corrected 🤔. Yunohost should create it under Calckey user.
Yes, I don't remember where they are but you can have the credentials/log as root or something. |
Rather than try to fix the live database (which I think may exceed my Postgresql skills), I am editing the 'db.sql' backup file. It seems it is NOT safe to do a global substitution of But there are many I then attempted to restore this. I ran 'pnpm run migrate' and this time, it did not report errors (it did give me a long list of queries, ending with "COMMIT", which seems like a successful result. I saved the output if you want to see it).
So then I restarted calckey, and I think I'm making progress, because I can access the application (screenshot), though it reports "An error has occurred". Visiting my settings shows me my profile. Initially, my banner image showed correctly, but my avatar was missing (apparently Calckeys shows colorbars for this). But after a few minutes, visiting my settings showed colorbars for my banner as well. In the log, I have a lot of errors like this:
The "id" hash this is showing looks a lot like fields I see in Trying to visit the timeline just gives me an error bird, like the screenshot below. This db.sql file is awkwardly large, so I tried creating a fresh Misskey site, backing that up, and examining that (MUCH smaller) db.sql file. However this didn't turn up anything interesting. Other than the So, I don't know what remains to be done, but the site is still broken. Less broken, which is encouraging, but still broken. |
Yes please, if it contains no personal information, that might be valuable later, at least to reproduce it and check if it's the same :) |
🎉 well done ! Can you check the status of calckey service (
Thanks a lot for your efforts in trying to make this work, we're making progress :) |
Here's output from the status query:
Note the Are these possibly "federation errors"? And I'm sorry, I spoke too soon about the output from |
Ok, we might need to full one (I don't remember the command) but that might be quite normal log I think, I have the same in my instance… In my case I understand it as federation errors yes. |
This may be a matter of degree. I get the impression that the Calckey instance can't find ANY of its resources. It reports a stream of users it can't find, and once I deleted cookies for the site from my browser, it goes back to trying to create a admin user. This fails if I try to use any of my old accounts, and I see this in the log:
So I try creating a new admin user, and I am able to log in. But no posts or notifications show up anywhere (interestingly, it is saying "No posts" now, rather than "An error occurred"). But as I am logged in as admin, I can now get to the control panel / dashboard: I poked around to find out what I might be able to see from the control panel, but the results are not encouraging: It doesn't list ANY users, except the newly-created admin. If I use the "Lookup" to look up user accounts and enter local accounts I know existed on the previous site, it reports not found. If I search for a remote account I know exists, also "User not found". If I put the fully qualified Fediverse ID for a local account Basically, aside from a few form fields, it doesn't seem to have gotten anything useful from the database -- it's like a new site with no content, it seems? If I go to the users panel, I can see the details for the new admin account (the only user it lists). If I hit the If I try to create a So it seems that it is colliding with the old data, but not actually able to use it. |
Interestingly, I do see that it tracked -4 users and -237 posts, which means that at the very least, chart data is preserved. |
So it is working to some degree 😅 |
Yes ? |
Currently we have instructions on how to migrate standard deployments of Misskey to Calckey here: https://codeberg.org/calckey/calckey/src/branch/develop/docs/migrate.md
However, someone recently asked how to do that for YunoHost, and sadly I don't have many pointers to give. If @oufmilo or someone else could elaborate on that process I'd be happy to add it to the official docs.
The text was updated successfully, but these errors were encountered: