Installer shows incorrect partition sizes, cannot install due to perceived ESP partition size #620

itaiferber · 2021-10-07T14:59:29Z

What Happened?

Attempting to install elementary OS 6 fails on my 2019 MacBook Pro, seemingly due to not being able to correctly read partition sizes on the disk. Attempting to install on this machine via the default "Erase Disk and Install" option fails with a "Installing elementary OS 6 Odin failed, possibly due to a hardware error." message, and attempting a "Custom Install" does not allow me to continue.

Steps to Reproduce

Boot into macOS Recovery to disable Secure Boot, booting from external partitions, etc.; delete existing partitions from disk
Boot into elementary OS 6 installer and attempt to "Erase Disk and Install"
1. Select the internal 1 TB NVME drive ("Apple SSD AP1024M, /dev/nvme0n1 1.0 TB")
2. Choose "Don't Encrypt" when prompted (though it doesn't matter — I've tried encrypting before and the failure is the same)
3. Hit "Could Not Install" error
  Screenshot
  I'm not sure whether the "pointer in FFI is null" error is unrelated, correlated, or causative, but the last error produced is related to ESP partition being too small
Go back with "Try Installing Again", this time selecting a "Custom Install"
1. "Modify Partitions" to create a 512 MB EFI partition, 16 GB swap partition, 64 GB root partition, and remaining home partition
  Screenshot
2. See that installer is incorrectly picking up on partition sizes (67.1 MB for EFI, 2.1 GB for swap, 8.6 GB for root, 114.3 GB for home, 875.5 GB unused)
  Screenshot
3. Attempt to select the partitions anyway (installation can't begin because EFI partition is too small)
  Screenshot

I've confirmed that booting into Demo Mode on the same drive shows the right partition sizes in GParted, as does booting into a GParted live USB.

Expected Behavior

"Erase and Install" should have succeeded at blowing away anything on disk and installing the OS
The partition selector should also show the same partition sizes as "Modify Partitions"

OS Version

6.x (Odin) [elementaryos-6.0-stable.20211005]

Software Version

Latest release (I have run all updates)

Log Output

I re-ran the above steps as-is (including recreating all partitions) this morning (the day after submission) and grabbed journalctl output from demo mode: journal.log

Hardware Info

This is on a 2019 MBP with a 1 TB internal drive and SIP and Secure Boot previously disabled.

The text was updated successfully, but these errors were encountered:

vjr · 2021-10-07T15:48:04Z

67 MB is about 1/8th of 512 MB which leads me to believe this is an issue in the installer/distinst not recognising 4k sector sizes?

itaiferber · 2021-10-08T13:41:24Z

I re-ran the above steps this morning and added log output from journalctl to the report.

itaiferber · 2021-10-08T13:43:52Z

@vjr Interesting. I didn't make the connection, but all of the perceived sizes are about 1/8th, + some offset. I'm assuming there's likely no way for me to work around this in the meantime by somehow explicitly telling the installer about the sector size?

vjr · 2021-10-08T14:14:17Z

I had tried working on a PR to support sector sizes a while back but could not test. The code might have changed a but...

Maybe if I have time this weekend I can revisit to see if I'm able to make any progress

itaiferber · 2021-10-08T14:15:44Z

@vjr Thanks! I'll keep my machine in this state for now — if you (or someone else) do have time to look into this, I'd be happy to provide any more info, or test out changes.

vjr · 2021-10-08T17:21:17Z

@itaiferber I've created a draft PR here: #621

I'll try to test (I dont have a 4k disk) maybe in a VM, but you are free to try my branch if you can.

I plan to boot an elementary 6 live ISO, install git and other tools/libraries as mentioned in the branch README notes and build and install the installer executable io.elementary.installer and try to run it.

vjr · 2021-10-09T20:33:27Z

@itaiferber I've been able to reproduce the problem with a qemu virtual disk with 4k sector size and also able to successfully install using these two branches:

https://github.com/vjr/installer/tree/fix-sector-size
https://github.com/vjr/distinst/tree/fix-sector-size

I had to also patch the distinst tool/library from the pop-os repos to work around the issue.

If you would try it, here are the (convoluted) steps I followed which I've typed from memory, so may not be completely accurate, you'll have to figure out any issues here:

Boot the elementary OS live ISO and pick the Try Demo mode to get to the desktop.
Open terminal and run the following commands:

sudo apt update
sudo apt install git elementary-sdk libgnomekbd-dev libpwquality-dev libdistinst-dev
sudo apt build-dep distinst
git clone https://github.com/vjr/installer.git
git clone https://github.com/vjr/distinst.git
cd distinst
git checkout fix-sector-size
make
sudo make install prefix=/usr
cd ../installer
git checkout fix-sector-size
meson build --prefix=/usr
cd build
ninja
sudo ninja install
sudo killall io.elementary.installer-daemon
sudo io.elementary.installer-daemon

Now run the installer by clicking the icon in the applications menu (dont run the executable from the terminal) and choose the "custom install" flow.

The "erase and install" option is failing, probably need to further patch for the LVM/default case, will do that later.

Looking forward to hearing back about your results!

itaiferber · 2021-10-09T20:41:19Z

@vjr Wow, thanks for such quick turnaround on this! I'll give those instructions a shot and let you know what I get.

itaiferber · 2021-10-09T21:34:47Z

@vjr I followed your instructions, and everything appears to have compiled and installed! Issues I ran into:

I hadn't checked out fix-vector-size in the installer directory, so initial attempts failed because the installer still didn't show the right info
Installation "failed" at the very end because efibootmgr could not set up the new installation in EFI (it appears that only macOS is able to set boot partitions, which is something I'll need to figure out and work around...)

But, I can confirm that I was able to boot up into elementary using rEFInd and am now setting up a user account. Thanks so much for the help!

vjr · 2021-10-10T10:04:54Z

@itaiferber so you have elementary OS installed and working on your mac now??? that's good to hear!

i missed the step about switching to the right git branch so edited that comment and also made a mention to not run the installer from the terminal but to run it by clicking the icon in the applications menu which causes a logout and launch of the installer - which perhaps also uses the right permissions which may or may not fix the efibootmgr failure - although i have not been able to verify this myself.

i'll continue to look into the failure i am seeing with the default "erase and install" flow (some error about not being able to create LVM partition/volume) maybe its some similar fix to the custom installation flow.

itaiferber · 2021-10-10T12:34:30Z

@vjr

so you have elementary OS installed and working on your mac now??? that's good to hear!

Yes! There's still a good amount of hardware support I need to figure out, but it is up and running. Thanks again 😄

which perhaps also uses the right permissions which may or may not fix the efibootmgr failure - although i have not been able to verify this myself.

Ah, should've been clearer about this — this isn't an elementary/installer failure. I haven't found efibootmgr to be able to write to this machine's NVRAM from any live USB or installation; the only tool I've gotten to work is macOS's own built-in bless tool. I think it's just missing hardware/firmware support, understandably.

i'll continue to look into the failure i am seeing with the default "erase and install" flow (some error about not being able to create LVM partition/volume) maybe its some similar fix to the custom installation flow.

For what it's worth, before managing the custom install, but with your branches, I did initially hit "Erase and Install" (just to see if it would fail in the usual way) and the installation did appear to actually begin. I didn't actually let it go through because I wanted to test the partitioning scheme, so I'm not sure whether it would have failed somewhere along the way, but at least I didn't immediately hit "Could not Install"...

vjr · 2021-12-02T01:01:24Z

I've now filed PR pop-os/distinst#278 to go along with the installer PR #621 if anyone wants to try them out and comment?

dinered · 2022-04-06T19:46:32Z

I've now filed PR pop-os/distinst#278 to go along with the installer PR #621 if anyone wants to try them out and comment?

Hello, @vjr !
I have Macbook and same problem.
Trying this #620 (comment)
but can't find fix-sector-size ...
test@MacBookPro0934d76e:~/distinst$ git checkout fix-sector-size error: pathspec 'fix-sector-size' did not match any file(s) known to git
I am not familiar with git at all, so could you be so kind and tell me what I am doing wrong?

vjr · 2022-04-07T04:23:57Z

Hello @dinered my bad I deleted the branch after my pr got merged but luckily github has a restore button!

Can you do a git pull in your distinst folder then retry the git checkout command and see if the rest of the steps still work?

dinered · 2022-04-07T21:49:19Z

Dear @vjr, first of all, thank you for your blazing fast answer!
I started with booting to Demo mode and find that space it is not enough to compile the new version.
After this I tried installing Elementary on a second 16Gb USB flash with the same result :)
The next one was 64Gb, I have installed Elementary (but there was no standard Installer in the system).
I did successfully every step from your list, except last two

sudo killall io.elementary.installer-daemon
Daemon wasn't found.
sudo io.elementary.installer-daemon
The command was not ended, cursor continued blink

Now installer starting, but doesn't show partitions at all.
I suppose, it is necessary to have the standard installer before compiling the new one, but can't check this right now, because Elementary boot looped to installer, and I don't know how to escape :)
Will try tomorrow, starting from scratch, if you don't tell me the easier way.

dinered · 2022-04-07T22:26:48Z

OMG, I just found that Odin doesn't support full disk encrypted custom installation, am I right?
Do you know, does eOS 7 beta support custom disk encryption?

vjr · 2022-04-08T02:32:21Z

Try adding a & to the daemon command to allow it to return to the command prompt?

I'm sorry I'm not familiar with the disk encryption part, maybe you can try the normal install for now and see if it works?

Note that elementary v7 is not beta yet, not even alpha level, only the base ubuntu 22.04 is beta so you should avoid it unless you really want to experiment either on your laptop or maybe in a VM.

dinered · 2022-04-14T05:53:11Z

@vjr , I did everything again, successful compile new version, now size is ok, but after custom install I had this errors -

INFO: getting device at /dev/nvmeon1
INFO: obtaining disk information from /dev/nvmeOn1
INFO: obtaining serial model from /dev/nvmeOn1
INFO: opening disk at /dev/nvmeOn1
INFO: obtaining partition informationgrom/dev/-
nvmeOn1p1
INFO: obtaining partition information from /dev/-
nvmeOn1p2
INFO: obtaining partition information from /dev/-
nvmeOn1p3
INFO: obtaining partition information from /dev/-
nvmeOn1p4
DEBUG: get block size for "/sys/class/block/nvmeOn1"
ERROR: libdistinst: pointer in FFI is null
ERROR: libdistinst: pointer in FFI is null
ERROR: libdistinst: pointer in FFI is null
DEBUG: get block size for "/sys/class/block/nvme0n1p1"
INFO: starting initializing step
INFO: Initializing
ERROR: config. squashfs: No such file or directory (os
error 2)
INFO: devices to modify: ["/dev/nvmeOn1p3"7
INFO: verifying if keyfiles have paths
ERROR: config.remove: unable to open file at "/cdrom/-
casper/filesystem.manifest-remove": No such file or
directory (os error 2)
INFO: obtaining list of physical volumes
INFO: volume map: {"/dev/sda2": Some("data")}
INFO: searching for device maps to deactivate
INFO: pvs: []
INFO: unmounting devices
ERROR: initializing error: unable to open file at
"/.
cdrom/casper/filesystem.manifest-remove":No such file
or directory (os error 2)
ERROR: errored while installing system: unable to open
file at "/cdrom/casper/filesystem.manifest-remove":No
such file or directory (os error 2)
INFO: Install error: unable to open file at
"/cdrom/-
casper/filesystem.manifest-remove": No such file or
directory (os error 2)

vjr · 2022-04-14T07:53:18Z

@dinered yes IIRC i had only done basic 4k block size support which i think doesn't work for logical volumes... can you try either standard install or custom install but create basic ext4 partition?

i have a couple of other PRs open (but from a while back) to try better 4k disk sector size support but i have to revisit them later on - in the mean time try a simple installation just to test?

dinered · 2022-04-14T09:11:48Z

@vjr , it was already basic ext4 partition. I have to create backup my macos partition to basic install, but i will try.

dinered · 2022-04-15T10:16:56Z

@vjr this is standard erase and install

INFO: getting device at /dev/nvmeon1
INFO: obtaining disk information from /dev/nvme0n1
INFO: obtaining serial model from /dev/nvme0n1
INFO: opening disk at /dev/nvmeOn1
INFO: obtaining partition information from /dev/nvme0n1p1
INFO: obtaining partition information from /dev/nvme0n1p2
INFO: specifying to write new table on /dev/nvme0n1
INFO: unmount all partitions on /dev/nvme0n1
DEBUG: get block size for
"/sys/class/block/nvme0n1"
DEBUG: get block size for
"/sys/class/block/nvme0n1"
DEBUG: get block size for
"/sys/class/block/nvmeOn1"
INFO: checking if 512:67870 overlaps
DEBUG: get block size for
"/sys/class/block/nvme0n1"
DEBUG: get block size for
"/sys/class/block/nvme0n1"
DEBUG: get block size for
"/sys/class/block/nvme0n1"
INFO: checking if 67871:61278831 overlaps
DEBUG: get block size for "/sys/class/block/nvmeOn1"
DEBUG: get block size for
"/sys/class/block/nvmeon1"
INFO: getting logical device named 'data_hugts'
DEBUG: get block size for
"/sys/class/block/data_huots"
DEBUG: get block size for
"/sys/class/block/data_husts"
DEBUG: get block size for
"/sys/class/block/data_husts"
DEBUG: get block size for
"/sys/class/block/data_hugts"
DEBUG: get block size for
"/sys/class/block/data_hugts"
DEBUG: get block size for
"/sys/class/block/data_hugts"
DEBUG: get block size for
"/sys/class/block/data_husts"
DEBUG: get block size for "/sys/class/block/data_hugts"
ERROR: libdistinst: pointer in FFI is null

jerano · 2022-08-30T19:14:06Z

Still an issue and can't try #620 (comment) because cannot install elementary-sdk due to unresolved dependencies - depends on libgala-dev, depends on libmutter-6-dev, depends on etc etc etc....

Trying to install "elementary OS 6.1 Jólnir"

PalGria · 2022-12-01T11:39:22Z

Just happened to me in a 12' inch retina 2015 macbook https://support.apple.com/kb/SP712

adamd9 · 2022-12-19T01:56:01Z

Hey @vjr,

Thanks for all your work on this issue! I have a 2020 MBP and this was a blocker for me trying out PopOS, I went through your original instructions, but came up against a couple of issues along the way so I thought I'd call out where I had trouble and provide updated instructions in case anybody else gives it a go.
Basically the two issues I found were:

The elementary repo wasn't available on the live USB to install the sdk, so I had to add it
Your installer fork was way behind the current popOS master fork (and I think you forked from elementaryOS anyway) - not surprising as I'm sure you've moved onto better things! I created a new fork based on the current popOS master.
While it appears they have merged your changes into the distinst popOS master, the logic appears to be broken and I still go the error about the EFS volume being smaller than 256MB. To get around this I just forked the latest master and removed this check completely (lazy I know, but I just wanted to get it working).

In the end, here's what my steps looked like to get things going (with links to my updated forks):

sudo add-apt-repository ppa:elementary-os/daily
sudo apt update
sudo apt install git elementary-sdk libgnomekbd-dev libpwquality-dev libdistinst-dev
sudo apt build-dep distinst
git clone https://github.com/adamd9/installer.git
git clone https://github.com/adamd9/distinst.git
cd distinst
make
sudo make install prefix=/usr
cd ../installer
git checkout sector-size-fix
meson build --prefix=/usr
cd build
ninja
sudo ninja install
sudo killall io.elementary.installer-daemon
sudo io.elementary.installer-daemon &

Per other earlier comments, the eftbootbgr fails to update at the end, but it doesn't really matter and you can still boot it (in the case of my Macbook Pro, by holding down the option key on restart and selecting "EFI manager" rather than the Mac startup disk

Hope that helps someone!

vjr · 2022-12-19T02:33:11Z

@adamd9 this is good stuff, thanks for the updates!

AdityaGarg8 · 2022-12-26T07:19:19Z

Installation "failed" at the very end because efibootmgr could not set up the new installation in EFI (it appears that only macOS is able to set boot partitions, which is something I'll need to figure out and work around...)

But, I can confirm that I was able to boot up into elementary using rEFInd and am now setting up a user account. Thanks so much for the help!

If you boot the ISO with efi=noruntime, the issue should be fixed.

This bug although has been fixed in newer kernels and Linux can now set boot partitions.

onefieryfern · 2023-03-06T13:35:39Z

Hey @vjr,

Thanks for all your work on this issue! I have a 2020 MBP and this was a blocker for me trying out PopOS, I went through your original instructions, but came up against a couple of issues along the way so I thought I'd call out where I had trouble and provide updated instructions in case anybody else gives it a go. Basically the two issues I found were:
* The elementary repo wasn't available on the live USB to install the sdk, so I had to add it

* Your **installer** fork was way behind the current popOS master fork (and I think you forked from elementaryOS anyway) - not surprising as I'm sure you've moved onto better things! I created a new fork based on the current popOS master.

* While it appears they have merged your changes into the **distinst** popOS master, the logic appears to be broken and I still go the error about the EFS volume being smaller than 256MB. To get around this I just forked the latest master and removed this check completely (lazy I know, but I just wanted to get it working).
In the end, here's what my steps looked like to get things going (with links to my updated forks):
* sudo add-apt-repository ppa:elementary-os/daily

* sudo apt update

* sudo apt install git elementary-sdk libgnomekbd-dev libpwquality-dev libdistinst-dev

* sudo apt build-dep distinst

* git clone https://github.com/adamd9/installer.git

* git clone https://github.com/adamd9/distinst.git

* cd distinst

* make

* sudo make install prefix=/usr

* cd ../installer

* git checkout sector-size-fix

* meson build --prefix=/usr

* cd build

* ninja

* sudo ninja install

* sudo killall io.elementary.installer-daemon

* sudo io.elementary.installer-daemon &
Per other earlier comments, the eftbootbgr fails to update at the end, but it doesn't really matter and you can still boot it (in the case of my Macbook Pro, by holding down the option key on restart and selecting "EFI manager" rather than the Mac startup disk

Hope that helps someone!

Hello there! I recently attempted to install Pop!_OS and had to use your guide to fix the installer, and I'd like to leave the list of commands which worked for me here, which is a bit different from yours.

sudo add-apt-repository ppa:elementary-os/daily
sudo apt update
sudo apt install git elementary-sdk libgnomekbd-dev libpwquality-dev libdistinst-dev
sudo apt build-dep distinst
git clone https://github.com/adamd9/installer.git
git clone https://github.com/adamd9/distinst.git
cd distinst
git checkout sector-size-fix
make
sudo make install prefix=/usr
cd ../installer
git checkout sector-size-fix
meson build --prefix=/usr
cd build
ninja
sudo ninja install
sudo killall io.elementary.installer-daemon
sudo io.elementary.installer &

derzahla · 2023-05-21T22:00:46Z

This is STILL a problem with the latest image nearly 2 years later? I take it elementaryOS project is dead..?

ladisnek · 2024-03-24T13:42:02Z

Hi, I think I have the same problem as was mentioned here with newest Elementary OS

https://github.com/orgs/elementary/discussions/494#discussion-6413044

I tried everything mentioned in this post and nothing has worked.

Am I right that it's the same issue? Or is it new issue?

Thanks in advance for any help, I am desperate.

vjr linked a pull request Oct 8, 2021 that will close this issue

Handle disk sector size #621

Open

vjr mentioned this issue Oct 9, 2021

Handle disk sector size pop-os/distinst#266

Merged

kbdharun mentioned this issue Jan 19, 2023

installer broken on MacBooks(?) Vanilla-OS/live-iso#94

Open

celticmagic mentioned this issue Aug 16, 2023

Installer doesn't tolerate SSD with 4KB LBA size (T10373) getsolus/packages#161

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Installer shows incorrect partition sizes, cannot install due to perceived ESP partition size #620

Installer shows incorrect partition sizes, cannot install due to perceived ESP partition size #620

itaiferber commented Oct 7, 2021 •

edited

Loading

vjr commented Oct 7, 2021

itaiferber commented Oct 8, 2021

itaiferber commented Oct 8, 2021

vjr commented Oct 8, 2021

itaiferber commented Oct 8, 2021

vjr commented Oct 8, 2021 •

edited

Loading

vjr commented Oct 9, 2021 •

edited

Loading

itaiferber commented Oct 9, 2021

itaiferber commented Oct 9, 2021

vjr commented Oct 10, 2021

itaiferber commented Oct 10, 2021

vjr commented Dec 2, 2021

dinered commented Apr 6, 2022

vjr commented Apr 7, 2022

dinered commented Apr 7, 2022 •

edited

Loading

dinered commented Apr 7, 2022

vjr commented Apr 8, 2022

dinered commented Apr 14, 2022

vjr commented Apr 14, 2022

dinered commented Apr 14, 2022

dinered commented Apr 15, 2022 •

edited

Loading

jerano commented Aug 30, 2022

PalGria commented Dec 1, 2022

adamd9 commented Dec 19, 2022

vjr commented Dec 19, 2022

AdityaGarg8 commented Dec 26, 2022

onefieryfern commented Mar 6, 2023 •

edited

Loading

derzahla commented May 21, 2023

ladisnek commented Mar 24, 2024

Installer shows incorrect partition sizes, cannot install due to perceived ESP partition size #620

Installer shows incorrect partition sizes, cannot install due to perceived ESP partition size #620

Comments

itaiferber commented Oct 7, 2021 • edited Loading

What Happened?

Steps to Reproduce

Expected Behavior

OS Version

Software Version

Log Output

Hardware Info

vjr commented Oct 7, 2021

itaiferber commented Oct 8, 2021

itaiferber commented Oct 8, 2021

vjr commented Oct 8, 2021

itaiferber commented Oct 8, 2021

vjr commented Oct 8, 2021 • edited Loading

vjr commented Oct 9, 2021 • edited Loading

itaiferber commented Oct 9, 2021

itaiferber commented Oct 9, 2021

vjr commented Oct 10, 2021

itaiferber commented Oct 10, 2021

vjr commented Dec 2, 2021

dinered commented Apr 6, 2022

vjr commented Apr 7, 2022

dinered commented Apr 7, 2022 • edited Loading

dinered commented Apr 7, 2022

vjr commented Apr 8, 2022

dinered commented Apr 14, 2022

vjr commented Apr 14, 2022

dinered commented Apr 14, 2022

dinered commented Apr 15, 2022 • edited Loading

jerano commented Aug 30, 2022

PalGria commented Dec 1, 2022

adamd9 commented Dec 19, 2022

vjr commented Dec 19, 2022

AdityaGarg8 commented Dec 26, 2022

onefieryfern commented Mar 6, 2023 • edited Loading

derzahla commented May 21, 2023

ladisnek commented Mar 24, 2024

itaiferber commented Oct 7, 2021 •

edited

Loading

vjr commented Oct 8, 2021 •

edited

Loading

vjr commented Oct 9, 2021 •

edited

Loading

dinered commented Apr 7, 2022 •

edited

Loading

dinered commented Apr 15, 2022 •

edited

Loading

onefieryfern commented Mar 6, 2023 •

edited

Loading