Mapping drive bays to locations

[Makowo]

We have 3 Drive enclosures (JBOD attached with LSI Raid controllers) with 12 drive bays each, we would like to map these to show the location of a drive instead of the /dev/sdx location if currently possible.

I've done some quick research and found https://serverfault.com/questions/1119649/how-to-name-drives-according-to-their-position-in-a-multi-bay-enclosure which suggests it should be possible. However, I am too inexperienced with Linux to fully understand how to implement something like this with ShredOS.

[PartialVolume]

That's a coincidence, I was just looking at how difficult it might be to replace the sdx into ../disk/by-id etc because of an existing request for this feature. Whether the pci address will sequentially correspond with a physical bay, I've no idea.

It's an interesting problem. I'll take a look at that link and have a think about it. I could easily match the pci address to the sdx address in the logs but presumably for this to be useful you need this in the GUI on completion of a drive having finished or failed while other drives are still wiping?

Anybody else got any possible solution

[Makowo]

We're trying to move from WipeOS (because why are we paying to wipe drives...) which had this feature. It replaced the /dev/sdx in the GUI with :0:1: being :shelf:bay: It didn't always provide this correctly but it was a massive help when looking for failed drives. It would be alright if it only shows after a wipe is failed/complete. if there's anything I can provide to help with this, please let me know.

[PartialVolume]

Let me see if I can add that information to the logs but also to the GUI. Can you tell me more about the hardware. Are the raid controllers in a PC connected via SATA to the jbod or fibre?

[Makowo]

I believe they're through sata\sff, but I will have to check tomorrow.

[PartialVolume]

@ggruber I have a another use for your test box.

[ggruber]

I see. And I have some other boxes ;-) to have a broader test base.

btw, why does the nwipe project not have discussions?

[ggruber]

@PartialVolume you might want to have a look at lsdiskowipe you find it @thelia

will try to integrate this physical path thing

[PartialVolume]

@ggruber

will try to integrate this physical path thing

We may already retrieve that information. Take a look at the call using 'readlink' we currently use it to extract bus type. But it will be retrieving the full path. The string would need extract/trimming/cropping.

We should also make any display of this by-id path optional i.e enable from command line option and in config screen.

Unless you are running a server with umpteen disc I don't think it's necessary, hence why I want to make it optional. Most people wiping a desktop or laptop don't need to know physical addressing.

[PartialVolume]

See line 618 in device.c

/* Scan the readlink results for bus types, i.e. USB or ATA
                 * Example: readlink
                 * /sys/block/sdd../devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.3/2-1.3:1.0/host6/target6:0:0/6:0:0:0/block/sdd
                 */

[ggruber]

I know about this.

And I agree with you about the unusual need for this feature.
my personal next #1 goal would be the secure erase feature.

as written before, I'll have a look into the path thing. Will have to check those readlink paths against what the adapter tools say about the slot number. sas2ircu 0 display @thelia will reveal the slot as seen from the adapter.
had a look:
enclosure 2 slot 7 corresponds to
pci-0000:03:00.0-sas-exp0x500056b36789abff-phy7-lun-0 from /dev/disk/by-path/
well, at least about slot 7 this looks good

readlink /sys/block/sdn
../devices/pci0000:00/0000:00:02.2/0000:03:00.0/host0/port-0:0/expander-0:0/port-0:0:58/end_device-0:0:58/target0:0:58/0:0:58:0/block/sdn

hmm

[ggruber]

I have a first implementation of a enclosure/slot detection for LSI/Avago/Broadcom HBAs in my lsdiskowipe tool:

[PartialVolume]

btw, why does the nwipe project not have discussions?

As a collaborator I don't have full control of all the features on nwipe's github pages. I'll contact Martijn van Brummelen who is the maintainer who can enable that feature. Also tagged @martijnvanbrummelen

[PartialVolume]

my personal next #1 goal would be the secure erase feature.

Totally agree, it's been a long time coming and it would be good to finally implement this.

Tomorrow, I'll open a discussion and break this feature down into the various sections that need to be written. Also how I see the interface changing very slightly to accommodate two new methods Secure Erase Basic and Secure Erase Enhanced and how we not only have a global method selection like we currently have but can override a method for any drive. So the user can select DOD3 globally for all drives but then change the SSDs to secure erase. To manually override the global wipe method for a given drive, you would use the selection keys to select the drive then type control+m to change the method, control+v to change the drive verification etc. Each drives details on the selection screen would need to incorporate these details, a example being [dod3,voff,R1,boff] meaning dod3 method, verification off, one round, blanking off. Also frozen drive info needs to be displayed, plus code to send the computer to sleep and wake-up necessary for unfreezing the drives on laptops and desktops.

Anyway that's the basics, I'll fill out the details tomorrow. I don't think it will be quite as large a project as adding pdf certificates but still a substantial amount of coding. Time will tell I guess.

[ggruber]

wouldn't be "if a drive supports secure erase then choose it, otherwise take "the common default" " a useful default?
then imho the selection per drive could wait a little

[PartialVolume]

Yes, we could have such an option. However, don't most spinning discs support secure erase anyway, I don't know about SAS but assume they do?, so most of the drives would be selected for secure erase with such an option, however you might want all spinning discs to be wiped the traditional way.

But yes, I see your point. We can built the feature in stages. Stage 1 get secure erase working as a method for SATA for starters, then NvMe and build on that.

[ggruber]

However, don't most spinning discs support secure erase anyway,

DEVICE  VENDOR         MODEL                SERIAL               FIRMWARE CAPACITY TRANSPORT IFSPEED   RPM   SECTSIZE HEALTH   SECTORS HOURS   TEMP %REMAIN ERRORS
sda     Dell           SSDSC2BB120G7R       PHDV8085029X150MGN   N201DL43 120 GB   sata      6 Gb/s    SSD   512/4096 PASSED   0       36945   37   100     0
sdb     Dell           SSDSC2BB120G7R       PHDV8086043N150MGN   N201DL43 120 GB   sata      6 Gb/s    SSD   512/4096 PASSED   0       36941   36   100     0
sdc     Seagate        ST1200MM0088         S400KXJ1             TT31     1.20 TB  sas       6 Gbps    10000 512      OK       0       37196   29           99
sdd     Seagate        ST1200MM0088         S400KXBX             TT31     1.20 TB  sas       6 Gbps    10000 512      OK       0       37196   29           376
sde     Seagate        ST1200MM0088         S400KXB2             TT31     1.20 TB  sas       6 Gbps    10000 512      OK       5789    37196   28           681
sdf     Toshiba        AL14SEB120N          9690A0XYF02C         DM06     1.20 TB  sas       6 Gbps    10000 512      ERROR    2892    32285   28           24
sdg     HP             MM1000FBFVR          9XGA0HGF0000C6347CLC HPD9     1.00 TB  sas       6 Gbps    7200  512      OK       0       47335   28           727
sdh     SanDisk        SSD PLUS 240GB       2205DH447504         UF4004RL 240 GB   sata      6 Gb/s    SSD   512      PASSED   0       1023    33   100     0
sdi     STEC           SDT5C-S200SS         C3M00GAM             K10T     200 GB   sas       6 Gbps    SSD   512      OK       0               52           0
sdj     STEC           Z16IZF2D-200UCT      STM000149A76         E125     200 GB   sas       6 Gbps    SSD   512      OK       0               46           0
sdk     Seagate        ST1000NX0423         W47060TK             TN03     1.00 TB  sata      6 Gb/s    7200  512      PASSED   0       35298   28           0
sdl     Seagate        ST1000NX0423         W472Z9C4             TN05     1.00 TB  sata      6 Gb/s    7200  512      PASSED   0       16773   25           0
sdm     Seagate        ST1000NX0423         W4719EKJ             TN04     1.00 TB  sata      6 Gb/s    7200  512      PASSED   0       37946   27           0
sdn     Seagate        ST1000NX0423         W4713RC0             TN04     1.00 TB  sata      6 Gb/s    7200  512      PASSED   0       37905   27           0
sdo     SE_HGST        DKR5E-J1R2SS         KZJRW8UG             BCBC     1.21 TB  sas       6 Gbps    10020 520      OK       0       57102   30           0
sdp     Toshiba        OCZ-TRION100         85PB52DZKMCX         SAFM11.2 240 GB   sata      6 Gb/s    SSD   512      PASSED           3563    24           0
sdq     NETAPP         X421_HCOBD450A10     PNHMT0GY             NA05     450 GB   sas       6 Gbps    10020 512      OK       0       11314   27           325199
sdr     HP             MM1000FBFVR          9XGA0GH70000C6347ECP HPD9     1.00 TB  sas       6 Gbps    7200  512      OK       0       53405   28           727
sds     Crucial/Micron CT500MX500SSD1       2205E604CA7C         M3CR043  500 GB   sata      6 Gb/s    SSD   512/4096 PASSED   0       348     25   100     0
sdt     SandForce      Corsair Force LS SSD 181379770001239600CD SBFM01.2 60.0 GB  sata      6 Gb/s    SSD   512      PASSED           1082    33   99
sdu     SanDisk        SSD PLUS 240GB       2205DH445208         UF4004RL 240 GB   sata      6 Gb/s    SSD   512      PASSED   1       799     32   100     0
sdv     Hitachi        HUC10603 CLAR300     PWKJ9DLD             C3B0     292 GB   sas       6 Gbps    10020 520      OK       0       41651   27           0
sdw     HP             MM1000FBFVR          9XGA0GWD0000C6347ENJ HPD9     1.00 TB  sas       6 Gbps    7200  512      OK       0       55478   29           1119
sdx     Hitachi        HUC10603 CLAR300     PWK8VW2D             C3B0     292 GB   sas       6 Gbps    10020 520      OK       0       40661   29           0
sdy     Hitachi        HUC10603 CLAR300     PWK8GAGD             C3B0     292 GB   sas       6 Gbps    10020 520      OK       0       40660   27           0
sdz     HP             MM1000FBFVR          9XGA0GVE0000C6347CN6 HPD9     1.00 TB  sas       6 Gbps    7200  512      OK       0       52579   29           1121
sdaa    Samsung        SSD 860 EVO 500GB    S3Z2NS0N637287J      RVT04B6Q 500 GB   uas-sata  6 Gb/s    SSD   512      PASSED   0       814     25   100     0
sdab    Samsung        SSD 860 EVO 1TB      S4CSNX0N704637Y      RVT04B6Q 1.00 TB  uas-sata  6 Gb/s    SSD   512      PASSED   0       165     28   100     0
nvme0   Samsung        SSD 980 PRO 250GB    S5GZNF0R126479R      2B2QGXA7 250 GB   nvme      8 GT/s x4 SSD   512      PASSED           7       47   100     0
nvme1   WD             WDS250G3X0C-00SJG0   2040AC801427         111110WD 250 GB   nvme      8 GT/s x4 SSD   512      PASSED           1016    47   100     0

 egrep -i '^sd[a-z]|security mode feature|model' /tmp/hdparmI | less
sda
        Model Number:       SSDSC2BB120G7R
sdb
        Model Number:       SSDSC2BB120G7R
sdc
sdd
sde
sdf
sdg
sdh
        Model Number:       SanDisk SSD PLUS 240GB
                Security Mode feature set
sdi
sdj
sdk
        Model Number:       ST1000NX0423
                Security Mode feature set
sdl
        Model Number:       ST1000NX0423
                Security Mode feature set
sdm
        Model Number:       ST1000NX0423
                Security Mode feature set
sdn
        Model Number:       ST1000NX0423
                Security Mode feature set
sdo
sdp
        Model Number:       OCZ-TRION100
                Security Mode feature set
sdq
sdr
sds
        Model Number:       CT500MX500SSD1
                Security Mode feature set
sdt
        Model Number:       Corsair Force LS SSD
                Security Mode feature set
sdu
        Model Number:       SanDisk SSD PLUS 240GB
                Security Mode feature set
sdv
sdw
sdx
sdy
sdz
sdaa
        Model Number:       Samsung SSD 860 EVO 500GB
                Security Mode feature set
sdab
        Model Number:       Samsung SSD 860 EVO 1TB
                Security Mode feature set

[ggruber]

regarding drive bay detection, just the example of thelia:

[ggruber]

so I see no generic way to get a drive position in a slot without a controller specific tool. I claim to have some experiences with different controllers, generic_raid_check still is to be published. It uses modules for each controller vendor, with specific implementations of data processing. (Originally to determine the drive status, but the slot numbering seems very similar to me).
If drives are connected to the mainboard connectors directly then you have no chance at all.
Only a HBA and/or HBA in conjunction with an expander chip in the backplane would allow to determine a slot position

[ggruber]

https://www.ibm.com/docs/en/linux-on-systems?topic=devices-secure-data-deletion-sas-drives
https://www.ibm.com/docs/en/linux-on-systems?topic=linuxonibm/liaau/t10pi.htm
https://www.thomas-krenn.com/en/wiki/Perform_a_SSD_Secure_Erase

should we move to a new discussion for the secure erase stuff?

[ggruber]

here the disk list with slot info for the thelia testmachine:

[ggruber]

and here I have a system with 3 controllers, 2 of them server a JBOD with finally 4 redundant connections.
They seem to interfere somehow, so far is no multipathing configured what should be done.
lsdiskowipe recognises dual ported disk, an now also lists the redundant paths from the controllers. A few expected paths are missing, there seem to be problems when the disks are accessed over different paths, And as my program also sanitized the disk names so that every physical disk is only show with exactly one sd* name, this list can be used to call nwipe with this name list on the CLI. The result is a wiping with real good performance.

CT = Controllertype (kernel module), C = Controller#, E=Enclosure, Slot