Board request: NanoPc CM3588 NAS #142
Comments
|
I too would like to know how support for the CM3588 would be added. |
|
You can use one of the existing platforms as a template. Board-specific code resides in the .DSC file and RockchipPlatformLib. The DTB should indeed be imported from Armbian's rk 6.1 branch. Right now it's only used for booting Linux, UEFI itself does not care about it. By the way, the PCIe code does not currently support bifurcation (which the CM3588 appears to be using), so things like NVME drives won't be usable in UEFI. |
|
Thank you for the response.
What exactly would be needed for PCIe bifurcation support? I ask because my primary use case for the board is as a NAS, with... you guessed it, NVMe drives. EDIT: OTOH, if it's only in UEFI where bifurcation doesn't work then perhaps it's not that much of a problem for the use case of a NAS? |
|
Bifurcation missing in UEFI means you can't boot from NVME. Maybe one of the slots would work at x1 link depending on how things are wired, but I don't know for sure. It shouldn't be difficult to implement. It mostly needs changes to the PCIe 3 PHY driver and some nice user-configurable settings for lane mapping. I haven't had the hardware to test this and hence no interest really, but I'm going to accept pull requests if anyone's interested in taking on this task. If your use-case is booting Linux with a device tree off SD/eMMC/USB, then you probably don't care about any of the above. |
|
I have begun simple work to get it ported to the device here. The fork/branch produces a bootable build of EDK2. This device is almost identical to the T6 which I used as the base. I don't particularly require NVMe boot however the first NVMe device did show as a boot option. I'm relatively new to the ARM scene and not very adept at reading device schematics however if I can reverse engineer the values from the T6 to understand how this is mapped I should be able to clean this up and submit a PR. @mariobalanica Maybe you can give me a simple example of how this mapping done? Otherwise, if anyone want's to take a crack at it you can build from my fork |
|
I tried your fork @navarro967, but my keyboard wouldn't get detected. If I was lucky I could get into the boot options by pressing Escape, but once I got in I couldn't navigate anything. Were you able to get this working on your board, or are you experiencing the same issue? |
|
I just got my hardware. |
@jjpe |
Not even if a wrong image is flashed, which at least in theory, could cause electrical damage by driving e.g. some I/O ports incorrectly? If that's true, then that is very useful to know indeed. |
Software wise they can't be bricked just like a regular pc, ofc physical damage can be done i.e plugging stuff where they don't belong but that rarely happens |
I was able to, I plugged my Keychron K10 into a USB A port and was able to enter the BIOS and make changes. |
|
I have tried 5 different Keyboards, and the only one that worked was a Microsoft Sculpt (notably, the only one which uses a wireless dongle). Every one of them worked, once booted into the OS. I couldn't find any error messages, the only thing i noticed was that all keyboards get power when plugged in, but only for a second before turning off again. |
|
Thanks for kicking this off @navarro967! I rebased your fork on the latest master to bring in the PCIe bifurcation support, and I can confirm that the firmware loads and sees my NVMe drive. I haven't actually managed to boot into an OS yet. I have a working Ubuntu installation on the eMMC -- I know it's working because if I remove the sd card with edk2 then it boots -- but selecting it in the boot configuration menu just flashes the screen black and brings me back to edk2. If anyone has suggestions about how to debug it, I'd be grateful. Otherwise, I'm going to poke around with GPIO settings. |
|
Got my NAS board today and tried @navarro967's fork, flashed to eMMC via the FriendlyWrt web flasher. Console is serial, connected to the three pin header. Looks pretty good, menu works fine, I switched from ACPI to DT and it boots into Grub from USB or microSD. I used a NixOS ISO ( After selecting the serial console in Grub and progressing to the kernel it gets stuck, though: Also tried an ISO from another distro with very similar results. Did not get any output on HDMI, but I didn't try very hard. Any idea on how to move on? |
|
I built the NixOS image from @ryan4yin: https://github.com/ryan4yin/nixos-rk3588/tree/main, the rock5a variant, needed a small patch: diff --git a/flake.nix b/flake.nix
index 5102bbc..ecffe1a 100644
--- a/flake.nix
+++ b/flake.nix
@@ -31,7 +31,7 @@
...
}: let
# Local system's architecture, the host you are running this flake on.
- localSystem = "x86_64-linux";
+ localSystem = "aarch64-linux";
pkgsLocal = import nixpkgs {system = localSystem;};
# The native system of the target SBC.
aarch64System = "aarch64-linux";
diff --git a/pkgs/mali-firmware/default.nix b/pkgs/mali-firmware/default.nix
index f2bb1c0..bc48e64 100644
--- a/pkgs/mali-firmware/default.nix
+++ b/pkgs/mali-firmware/default.nix
@@ -7,7 +7,7 @@ stdenv.mkDerivation {
version = "g21p0-01eac0";
src = fetchurl {
- url = "https://github.com/JeffyCN/mirrors/raw/e08ced3e0235b25a7ba2a3aeefd0e2fcbd434b68/firmware/g610/mali_csffw.bin";
+ url = "https://github.com/JeffyCN/mirrors/raw/libmali/firmware/g610/mali_csffw.bin";
hash = "sha256-jnyCGlXKHDRcx59hJDYW3SX8NbgfCQlG8wKIbWdxLfU=";
};
Copied to SD; throws errors due to hardware mismatch but boots fully with network working (on serial console, did not try HDMI). |
Could you elaborate on the steps you took to get this working? I am unable to reproduce. The rawEfiImage fails to build, and the sdImage seems to have a broken partition table (I cannot mount the second partition, and the first one is empty) |
|
My local system isn't an aarch64-linux, so I only changed the mali firmware url. (But it doesn't look like it matters; Then I run (I've been struggling with updating to a more recent kernel, but that's another story...) |
|
@jneem What kernel versions worked for you? |
I tried booting the second two options (armbian/linux-rockchip and mainline 6.10) using armbian (instead of nixos) to build an sdcard image. The bootable armbian sdcards skip edk2 and they boot with working ethernet. |
|
Btw, I have an updated branch here. Compared to @navarro967 's branch, it's based on a more recent master and it comes with some updates to the pin definitions (but I didn't actually observe any differences caused by these changes) |
|
Thanks for your work @jneem! Did you use mainline with just the armbian config or did you also apply their kernel patches? I have tried to build the 6.10.{5,9} Kernels with the armbian patches and a modified version of their config (with compressed firmware loading enabled), but haven't been able to successfully build it yet. I am pretty sure it is an issue with my build process though. I have also tried the armbian 6.1.75 and the friendlyarm 6.1.57 kernels, both boot but no Ethernet. The chip doesn't show in lspci, i haven't done more digging yet. Curiously the friendlyarm kernel with almost the same config (i had to enable efi and dmiid) and the same dts has working ethernet when used in the vendor debian 12 image. I have made a flake to test nixos on the cm3588 a while ago, but just now gotten around to cleaning it up. Here is a link for anyone that's interested. |
|
I've been trying without the armbian patches so far, but maybe I should give them a try. Btw, 6.11 now has mainline support for the cm3588! I'm building that now. Working ethernet is a priority for me, but I was hoping to get a new kernel booting before digging into the ethernet issue... |
If it's the same issue I had, it's because you need to set |
|
I got 6.11 working (with ethernet, but no HDMI for some reason) on my cm3588. Here is the minimal config needed: boot.kernelPackages = pkgs.linuxPackagesFor pkgs.linux_latest;
hardware = {
enableRedistributableFirmware = true;
deviceTree = {
enable = true;
name = "rockchip/rk3588-friendlyelec-cm3588-nas.dtb";
};
}; |
The Crypto instructions on RK3588 are disabled by default. Whether they are enabled in your environment depends on what build of BL31 is being used. See this patch for reference: https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/29363 |
Amazing! All that fiddling with armbian forks and custom configs, and it turns out that latest mainline just works! (For anyone following along, it's important here to be using |
|
I am trying to flash the firmware from this branch (https://github.com/jneem/edk2-rk3588/tree/cm3588), however, I am unable to enter the boot options when pressing escape at the boot splash screen. I've tried 2 different keyboards. It boots into openmediavault by default no matter what I do. Any suggestions? |
|
It looks like only one USB port was being powered up (the upper USB 3.0 port). The latest commit should enable them all. |
|
For the NixOS users here, how did you cross compile jneem's EDK2 branch? |
|
@jneem Thanks a lot for your work! 🤗
Until someone actually answers your question .. I did not want to trial and error the required tools and went the Debian route (on Enable Create In Debian: Find Have not tried the UEFI image yet. I also have a |
in EFI I have ACPI/DeviceTree set to I'm booting NixOS from emmc and have just installed and configured four nvme drives: Running stock linux-6.11 with: |
|
Thank you, that will be most helpful. One tricky aspect: So one (hopefully) last question: How did you manage to install the OS to the emmc drive? |
With EFI set to
From https://download.friendlyelec.com/CM3588 found on https://wiki.friendlyelec.com/wiki/index.php/CM3588 I downloaded In Debian I installed
|
|
This definitely helped, as it allowed me to partition and format the nvme and internal emmc drives, and then install NixOS on the emmc drive. The only thing I'm wondering is: how did you blindly configure the network? I'm asking because after selecting a NixOS generation to boot, the screen (expectedly) goes blank, the keyboard lights stay lit, but when I try to log in with the root account (for which a password was entered during install), nothing seems to happen. And I can't tell the difference except for running |
First thing I did was also power off the system to verify successful login. I use During install, I use a stripped down For the interface: I constructed a oneliner to fetch the name of the interface, and assign an ip to it:
I carefully typed the command and verified success with ping running on my laptop. I then could ssh in with the non-root account already configured during install, Did you try I later learned of the kernel parameter |
I'll post my configuration for clarity, as viewed from the booted Debian SD card image. I use 2 partitions on the emmc drive. mkdir -p /mnt-nixos/boot \
&& mount /dev/disk/by-label/NIXROOT /mnt-nixos \
&& mount /dev/disk/by-label/NIXBOOT /mnt-nixos/boot/ -o umask=077,fmask=0077,dmask=0077 \
&& systemctl daemon-reload
The reason I made these formatting choices is that the NixOS manual uses them.
{ config, lib, pkgs, ... }:
{
imports =
[ # Include the results of the hardware scan.
./hardware-configuration.nix
];
boot.kernelPackages = pkgs.linuxPackages_latest;
boot.loader = {
efi = {
canTouchEfiVariables = true;
};
systemd-boot = {
enable = true;
configurationLimit = 10;
};
};
# TODO: configure ZFS
# boot.supportedFilesystems.zfs = true;
# boot.initrd.supportedFilesystems.zfs = true;
# boot.zfs.package = pkgs.zfs_unstable;
# services.zfs = {
# autoScrub.enable = true;
# autoSnapshot.enable = true;
# };
networking.hostId = "54e492c4"; # See https://nixos.org/manual/nixos/unstable/options#opt-networking.hostId
nixpkgs.config.allowBroken = true; # allow broken ZFS for now
hardware.enableRedistributableFirmware = true;
hardware.deviceTree = {
enable = true;
name = "rockchip/rk3588-friendlyelec-cm3588-nas.dtb";
};
networking.hostName = "cumulus"; # Define your hostname.
# Set your time zone.
time.timeZone = "Europe/Amsterdam";
# Configure network proxy if necessary
# networking.proxy.default = "http://user:password@proxy:port/";
# networking.proxy.noProxy = "127.0.0.1,localhost,internal.domain";
i18n.defaultLocale = "en_US.UTF-8";
console = {
# font = "Lat2-Terminus16";
keyMap = "us";
# useXkbConfig = true; # use xkb.options in tty.
};
# Configure keymap in X11
services.xserver.xkb.layout = "us";
# Define a user account. Don't forget to set a password with ‘passwd’.
users.users.j = {
isNormalUser = true;
initialPassword = "cumulus"; # just a placeholder
extraGroups = [ "networkmanager" "wheel" ]; # Enable ‘sudo’ for the user.
packages = with pkgs; [];
};
# List packages installed in system profile. To search, run:
# $ nix search wget
environment.systemPackages = with pkgs; [
git
vim
# wget
];
# Enable the OpenSSH daemon.
services.openssh.enable = true;
services.openssh.settings.PasswordAuthentication = true; # TODO disable
networking = { # I also tried the eth0 interface, as that's how it's enumerated in Debian. Alas, to no avail.
interfaces.enP4p65s0 = {
ipv4.addresses = [{
address = "192.168.1.200";
prefixLength = 24;
}];
ipv6.addresses = [{
address = "fdf1:c31d:8bf2::1e6";
prefixLength = 64;
}];
};
defaultGateway = {
address = "192.168.1.1";
interface = "enP4p65s0" ;
};
defaultGateway6 = {
address = "fdf1:c31d:8bf2::1";
interface = "enP4p65s0" ;
};
};
# Open ports in the firewall.
networking.firewall.allowedTCPPorts = [ 22 ]; # shouldn't be necessary but can't hurt during setup.
# This option defines the first version of NixOS you have installed on this particular machine,
# and is used to maintain compatibility with application data (e.g. databases) created on older NixOS versions.
#
# Most users should NEVER change this value after the initial install, for any reason,
# even if you've upgraded your system to a new NixOS release.
#
# This value does NOT affect the Nixpkgs version your packages and OS are pulled from,
# so changing it will NOT upgrade your system - see https://nixos.org/manual/nixos/stable/#sec-upgrading for how
# to actually do that.
#
# This value being lower than the current NixOS release does NOT mean your system is
# out of date, out of support, or vulnerable.
#
# Do NOT change this value unless you have manually inspected all the changes it would make to your configuration,
# and migrated your data accordingly.
#
# For more information, see `man configuration.nix` or https://nixos.org/manual/nixos/stable/options#opt-system.stateVersion .
system.stateVersion = "24.11"; # Did you read the comment?
}I started out with As for the drives, I mount them by uuid (which I also do on my other nixos hosts): I have 4 identical NVMe drives so it seems like there's less chance of confusion when mounting by label.
# Do not modify this file! It was generated by ‘nixos-generate-config’
# and may be overwritten by future invocations. Please make changes
# to /etc/nixos/configuration.nix instead.
{ config, lib, pkgs, modulesPath, ... }:
{
imports = [
(modulesPath + "/installer/scan/not-detected.nix")
];
boot.initrd.availableKernelModules = [ "nvme" "usbhid" ];
boot.initrd.kernelModules = [ ];
boot.kernelModules = [ ];
boot.extraModulePackages = [ ];
fileSystems."/" = {
device = "/dev/disk/by-uuid/729b1082-27d9-45cb-88da-f2117f2d3ec9";
fsType = "ext4";
options = [ "lazytime" ];
};
fileSystems."/boot" = {
device = "/dev/disk/by-uuid/08D8-4A5D";
fsType = "vfat";
options = [ "umask=0077" "fmask=0077" "dmask=0077" ];
};
swapDevices = [ ];
# Enables DHCP on each ethernet and wireless interface. In case of scripted networking
# (the default) this is the recommended approach. When using systemd-networkd it's
# still possible to use this option, but it's recommended to use it in conjunction
# with explicit per-interface declarations with `networking.interfaces.<interface>.useDHCP`.
networking.useDHCP = lib.mkDefault true;
# networking.interfaces.eth0.useDHCP = lib.mkDefault true;
nixpkgs.hostPlatform = lib.mkDefault "aarch64-linux";
}I installed nixos using {
description = "NixOS Configuration Flake (cumulus)";
inputs = {
# NixOS official package source, using the `nixos-unstable` branch
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
home-manager.url = "github:nix-community/home-manager";
home-manager.inputs.nixpkgs.follows = "nixpkgs";
};
outputs = inputs@{ self, nixpkgs, home-manager, ... }: {
nixosConfigurations.cumulus = nixpkgs.lib.nixosSystem {
system = "aarch64-linux";
modules = [
# Import the old config file so it will still take effect
./configuration.nix
home-manager.nixosModules.home-manager
{
home-manager.useGlobalPkgs = true;
home-manager.useUserPackages = true;
home-manager.users.j = import ./home.nix;
# Optionally, use `home-manager.extraSpecialArgs`
# to pass arguments to ./home.nix
}
];
};
};
}I also tried disabling home manager for now, but that turns out not to make a difference.
Once I can log in, this will be most useful, thanks.
This is where things currently break for me. I tried logging in with both
Just tried that. The outcome is that it prints a few lines (apologies for the quality): After that the screen goes blank as usual. |
This will not work if you Can you show the contents of your
I'm surprised by the
Nice hostname 😎 I named my company Humilis, after Cumulus Humilis.
I'm using
I do however, disable both firewall and dhcp: Likely does not matter.
Agreed. Although the NVMe drives are not in play yet.
Yeah home-manager has no part in the OS itself. I have my home-manager entirely in my home directory btw, but unrelated.
Nothing obvious. It is a long shot but could be missing files in If you can't make it work I'll recreate the stripped-down
I guess this requires upstream HDMI support then. Next step could be, find the patch that enables HDMI in Armbian and include that in your NixOS kernel stanza. |
Indeed, what I did to gather this information was:
In particular, the partitions I'm mounting were created just before I created the first NixOS generation with the first execution step 4. In order to create and format the partitions with the proper flags I followed the steps outlined in the NixOS manual. Except I used a flake, which should work just as well as direct usage of
In particular,
If my assumption is correct that
Yeah it's definitely a little weird - also that there's no
Thank you! That's also where I got the name from :)
Did you only use
It's worth a shot, so I tried it. But yeah, these options (including disabling DHCP) doesn't seem to make a difference.
That's starting to look pretty good, thanks.
In the meantime I'll explore this option. |
As you are posting this sequence of commands again: The directory The only reason that second
Yes, I install using flakes the same way you do.
Looks good. For reference:
I have - stripped ipv6 details - but as I had the wrong interface name it was of no use:
|
Yeah IIRC the first time I had to mkdir that directory manually on Therefore my reasoning was that the empty But maybe you know something I don't know, and more is needed than just making it available to
I get similar output for both queries: So I'm thinking that part is probably ok.
Thank you for taking the effort. Bafflingly, and the strongest hint yet something else might be going on here: Of course, if the mounting above step truly is problematic as it is, then perhaps that is where (part of ) the solution lies. I'm going to have another look at the nixos manual, see if there's something I missed. |
Thing is, Create directory: List contents of Mount filesystem at List contents of The directory To make So I'm really just nitpicking as you kept posting the command sequence that created a directory that you could not use.
In Armdebian, can you post the output of |
Sure, here's the output: |
This seems a successful boot. Can you try to logon?
Network:
Could it be that your address is missing trailing
|
When I try to log into the root account and then execute However, after I log in, I execute So something is still off with the network config in particular.
I tried adding the What I'll try next is adding the subnet mask to both the EDIT: I've finally managed to ssh into the device using my non-root account, so thank you for the help! |
Glad you made it work! I would not bother with |
|
Turns out I already had those blocks in my config. If I ever meet you in meatspace, I definitely owe you a beer :) |
|
There's 1 final issue I need to tackle: The nixos config provides LAN access to I can ssh into it, but no internet access, making updating the system a bit difficult. So I create a default route using The only issue is, at the moment it's necessary that I execute it on every reboot, which is undesirable. networking = {
dhcpcd.enable = false;
firewall.enable = false;
hostName = "cumulus";
useDHCP = false;
useNetworkd = true;
interfaces.enP4p65s0.ipv4.routes = [{ # I was hoping this would do the trick
address = "192.168.1.145/24";
prefixLength = 24;
via = "192.168.1.1/24";
}];
};But that just borks the LAN connection, necessitating a reinstall from Armbian. Any idea how I can "nixify" the EDIT: I also tried this, but without success: systemd.services.internetAccess = {
wantedBy = [ "multi-user.target" ];
after = [ "network.target" ];
description = "Add internet access.";
path = [pkgs.bash pkgs.iproute];
script = ''
ip route add default via 192.168.1.1 dev enP4p65s0
'';
serviceConfig = {
Type= "oneshot";
User = "root";
Restart = "no";
};
}; |
|
@jjpe you set |
Everything I mentioned in my last post is with this block in the configuration: systemd = {
network = {
enable = true;
networks = {
enP4p65s0 = {
address = [ "192.168.1.145/24" ];
matchConfig = { Name = "enP4p65s0"; Type = "ether"; };
gateway = [ "192.168.1.1/24" ];
};
};
};
};And this does allow me to ssh in from another host on my LAN. But it doesn't provide internet access for the CM3588, necessitating the manual execution after every boot of That's a bit... suboptimal for a server, especially so since none of my other hosts need any manual network configuration at all. |
The gateway must not have the |
|
Removing the gateway subnet mask made the difference. Thanks, now the real configuration can begin :) |
|
Somewhat related to this thread: does anyone happen to know if the same Looking at the website, it looks exactly like the CM3588 except with more RAM and possibly higher capacity emmc. But I also know that looks can be deceiving when it comes to hardware, so I'm not making any assumptions. |
Yes, mine is a |
|
@sanderhollaar Thank you for this info, that helps a lot! Aside from my |
|
First of all, this is really great work. I'll get a CM3588 next week and am really looking forward to having an alternative to the Debian BSP. However, from the Collabora-Link, it looks like running this board in a mainline 6.11 kernel will not support any of the video de-/encoding features of the board, right? I was hoping I could run Jellyfin there, because they support
Is it possible to boot the verndor's BSP Debian kernel as a workaround (if that's needed)? |
There won't be any mature replacement for RKMPP & RKRGA in the mainline anytime soon, so Jellyfin rewrote RKMPP support in FFmpeg from scratch. If you need normal transcoding functionality, please stay away from the mainline. https://jellyfin.org/docs/general/administration/hardware-acceleration/rockchip |
Do you happen to know the reason for this? |
Because third-party developers do not have enough resources and time to develop extremely complex VPU drivers in the mainline full-time, and because V4L2 cannot meet Rockchip's actual product needs, they will not spend any time in upstream V4L2. Read through this ticket. jellyfin/jellyfin-ffmpeg#34 |
|
Not sure if this question belongs here, but has anyone gotten higher speeds than 100Mb/s on the NIC? I'm running nixos on the 6.11 kernel, and for whatever reason the NIC decides to run in 100Mb/s mode. The only thing I can think of is that it's some limitation of the DTS file in the 6.11 kernel, kind of like how HDMI isn't working either. If anyone has any info on this, please let me know! :) EDIT: Today I Learned that I had a CAT5 cable in my home network setup...... |
What would be required to add support for the FriendlyElec CM3588 NAS kit board? I have very little experience in this area, but I am motivated to get this implemented. I do own this board myself, so I would be able to perform testing on it.
What I've figured out myself is that the device tree (dts) file needs to be included here. The proper device tree for this board is probably this one.
Are there any more files that need to be included for proper support?
Looking forward for input!
The text was updated successfully, but these errors were encountered: