Initial yocto support

[bhargavdas]

Yocto recipes for mtda.

Yocto Compatibilty
Release: v3.4 , honister (since the overriding syntax has changed and are not backward compatible)

What's been done

• Created new recipes for some of the python libraries and also for mtda.
• New image named mtda-image, based on core-image-minimal.
• Services are pushed to rootfs from the recipes unlike isar.

Status

• Work in progress.
• Though the mtda recipe's PV is 0.16 but currently the recipe is using the tip of the master branch (for development). Also many things are yet to be updated to align with latest changes.

What works

• mtda-cli
• mtda service
• Other basic features

What doesn't work

• Switching of mass storage device from target to host (and vice versa).

Tested on

• imx6dlsdb

Maintenance Strategy (suggestion)

• Update the recipes once a -tc1 release tag is assigned, and start testing.

Thing to review now for Maintainers

• Check the licenses in the recipes.
• Is the maintenance strategy acceptable ?
• Review the layer name if it is appropriate.
• Review the recipes.
• Check for unwanted bits.
• Suggestions for improvements.

[vj-kumar]

@bhargavdas Wondering if we can start with a proven mtda platform for the initial support. Beaglebone black might be a good candidate to start with. All you need is the poky repo.

Starting with such a platform will give us some idea on how much of things we can reuse, and how we can align our existing implementation to achieve that.

[vj-kumar]

@bhargavdas Wondering if we can start with a proven mtda platform for the initial support. Beaglebone black might be a good candidate to start with. All you need is the poky repo.

Starting with such a platform will give us some idea on how much of things we can reuse, and how we can align our existing implementation to achieve that.

https://github.com/vj-kumar/mtda/blob/vijai/yocto/kas/yocto/bbb.yml

Initial yocto bbb yml file. Builds fine.

[bhargavdas]

@bhargavdas Wondering if we can start with a proven mtda platform for the initial support. Beaglebone black might be a good candidate to start with. All you need is the poky repo.

Starting with such a platform will give us some idea on how much of things we can reuse, and how we can align our existing implementation to achieve that.

I agree, but I don't have a beaglebone black to test. Though I can build an image using this layer and ask someone to test it.

[vj-kumar]

@bhargavdas Wondering if we can start with a proven mtda platform for the initial support. Beaglebone black might be a good candidate to start with. All you need is the poky repo.
Starting with such a platform will give us some idea on how much of things we can reuse, and how we can align our existing implementation to achieve that.

I agree, but I don't have a beaglebone black to test. Though I can build an image using this layer and ask someone to test it.

I have one and I can lend it to you.

[bhargavdas]

@bhargavdas Wondering if we can start with a proven mtda platform for the initial support. Beaglebone black might be a good candidate to start with. All you need is the poky repo.
Starting with such a platform will give us some idea on how much of things we can reuse, and how we can align our existing implementation to achieve that.

I agree, but I don't have a beaglebone black to test. Though I can build an image using this layer and ask someone to test it.

I have one and I can lend it to you.

Sounds good :)

[bhargavdas]

@vj-kumar all upstream recipe dropped from layer
91f3168

initial sanity checks working as expected.

[bhargavdas]

@bhargavdas Wondering if we can start with a proven mtda platform for the initial support. Beaglebone black might be a good candidate to start with. All you need is the poky repo.
Starting with such a platform will give us some idea on how much of things we can reuse, and how we can align our existing implementation to achieve that.

I agree, but I don't have a beaglebone black to test. Though I can build an image using this layer and ask someone to test it.

I have one and I can lend it to you.

Sounds good :)

The layer is building with beaglebone-yocto image . At last moment build of sd-mux-ctrl failed. :( Checking
Since this layer prepared sometime back, few services like mtda-usb-functions need to removed as it is merge now with mtda itself.

[bhargavdas]

@bhargavdas Wondering if we can start with a proven mtda platform for the initial support. Beaglebone black might be a good candidate to start with. All you need is the poky repo.
Starting with such a platform will give us some idea on how much of things we can reuse, and how we can align our existing implementation to achieve that.

I agree, but I don't have a beaglebone black to test. Though I can build an image using this layer and ask someone to test it.

I have one and I can lend it to you.

Sounds good :)

The layer is building with beaglebone-yocto image . At last moment build of sd-mux-ctrl failed. :( Checking Since this layer prepared sometime back, few services like mtda-usb-functions need to removed as it is merge now with mtda itself.

sd-mux-ctrl build and warnings fixed.

[bhargavdas]

Initial checks on beaglebone black looks fine.

service

● mtda.service - Multi-Tenant Device Access
     Loaded: loaded (/lib/systemd/system/mtda.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2022-05-05 04:25:20 UTC; 5s ago
   Main PID: 294 (python3)
      Tasks: 1 (limit: 1130)
     Memory: 15.0M
     CGroup: /system.slice/mtda.service
             └─294 python3 /usr/bin/mtda-cli -d -n

May 05 04:25:20 mtda systemd[1]: Started Multi-Tenant Device Access.
root@mtda:~# 

mtda-cli help

root@mtda:~# mtda-cli help
usage: mtda [options] <command> [<args>]

The most commonly used mtda commands are:
   command   Send a command (string) to the device
   console   Interact with the device console
   getenv    Get named variable from the environment
   monitor   Interact with the device monitor console (if any)
   target    Power control the device
   setenv    Set named variable to specified value in the environment
   storage   Interact with the shared storage device
   usb       Control USB devices attached to the device

root@mtda:~# 

Next tasks before dropping WIP tag

• Cleanup layer and reuse bits directly from this repo as @vj-kumar suggested
• Check and validate other features
• Remove License warnings during build

[bhargavdas]

Somehow /sys/class/gpio directory is not present in the system, hence mtda service fails :(

root@mtda:~# ls /sys/class/gpio
ls: /sys/class/gpio: No such file or directory
root@mtda:~# lsmod
Module                  Size  Used by
snd_soc_simple_card    16384  0
snd_soc_simple_card_utils    24576  1 snd_soc_simple_card
snd_soc_davinci_mcasp    32768  2
snd_soc_ti_udma        16384  1 snd_soc_davinci_mcasp
snd_soc_ti_edma        16384  1 snd_soc_davinci_mcasp
snd_soc_ti_sdma        16384  1 snd_soc_davinci_mcasp
snd_soc_hdmi_codec     16384  1
snd_soc_core          204800  7 snd_soc_davinci_mcasp,snd_soc_hdmi_codec,snd_soc_simple_card_utils,snd_soc_ti_sdma,snd_soc_ti_edma,snd_soc_ti_udma,snd_soc_simple_card
snd_pcm_dmaengine      16384  1 snd_soc_core
snd_pcm               106496  4 snd_soc_davinci_mcasp,snd_pcm_dmaengine,snd_soc_hdmi_codec,snd_soc_core
snd_timer              32768  1 snd_pcm
snd                    65536  4 snd_soc_hdmi_codec,snd_timer,snd_soc_core,snd_pcm
soundcore              16384  1 snd
sch_fq_codel           20480  2
fuse                  118784  1
configfs               40960  1
root@mtda:~# 

[bhargavdas]

Somehow /sys/class/gpio directory is not present in the system, hence mtda service fails :(

root@mtda:~# ls /sys/class/gpio
ls: /sys/class/gpio: No such file or directory
root@mtda:~# lsmod
Module                  Size  Used by
snd_soc_simple_card    16384  0
snd_soc_simple_card_utils    24576  1 snd_soc_simple_card
snd_soc_davinci_mcasp    32768  2
snd_soc_ti_udma        16384  1 snd_soc_davinci_mcasp
snd_soc_ti_edma        16384  1 snd_soc_davinci_mcasp
snd_soc_ti_sdma        16384  1 snd_soc_davinci_mcasp
snd_soc_hdmi_codec     16384  1
snd_soc_core          204800  7 snd_soc_davinci_mcasp,snd_soc_hdmi_codec,snd_soc_simple_card_utils,snd_soc_ti_sdma,snd_soc_ti_edma,snd_soc_ti_udma,snd_soc_simple_card
snd_pcm_dmaengine      16384  1 snd_soc_core
snd_pcm               106496  4 snd_soc_davinci_mcasp,snd_pcm_dmaengine,snd_soc_hdmi_codec,snd_soc_core
snd_timer              32768  1 snd_pcm
snd                    65536  4 snd_soc_hdmi_codec,snd_timer,snd_soc_core,snd_pcm
soundcore              16384  1 snd
sch_fq_codel           20480  2
fuse                  118784  1
configfs               40960  1
root@mtda:~# 

@chombourger @vj-kumar , Do I need to add / load any modules or dts entries ? Need a starting point to start debugging.

[vj-kumar]

Somehow /sys/class/gpio directory is not present in the system, hence mtda service fails :(

root@mtda:~# ls /sys/class/gpio
ls: /sys/class/gpio: No such file or directory
root@mtda:~# lsmod
Module                  Size  Used by
snd_soc_simple_card    16384  0
snd_soc_simple_card_utils    24576  1 snd_soc_simple_card
snd_soc_davinci_mcasp    32768  2
snd_soc_ti_udma        16384  1 snd_soc_davinci_mcasp
snd_soc_ti_edma        16384  1 snd_soc_davinci_mcasp
snd_soc_ti_sdma        16384  1 snd_soc_davinci_mcasp
snd_soc_hdmi_codec     16384  1
snd_soc_core          204800  7 snd_soc_davinci_mcasp,snd_soc_hdmi_codec,snd_soc_simple_card_utils,snd_soc_ti_sdma,snd_soc_ti_edma,snd_soc_ti_udma,snd_soc_simple_card
snd_pcm_dmaengine      16384  1 snd_soc_core
snd_pcm               106496  4 snd_soc_davinci_mcasp,snd_pcm_dmaengine,snd_soc_hdmi_codec,snd_soc_core
snd_timer              32768  1 snd_pcm
snd                    65536  4 snd_soc_hdmi_codec,snd_timer,snd_soc_core,snd_pcm
soundcore              16384  1 snd
sch_fq_codel           20480  2
fuse                  118784  1
configfs               40960  1
root@mtda:~# 

@chombourger @vj-kumar , Do I need to add / load any modules or dts entries ? Need a starting point to start debugging.

Is CONFIG_GPIO_SYSFS kernel config set? zcat /proc/config.gz|grep GPIO

[bhargavdas]

Is CONFIG_GPIO_SYSFS kernel config set? zcat /proc/config.gz|grep GPIO

After multiple rebuilds (with different kernels ) I found that CONFIG_GPIO_SYSFS is not present in the latest linux-yocto kernel 5.15.36 but it is present in 5.10.63.

Initial gpio control tests mtda-cli target on/off seems work fine now with 5.10 kernel.
Now looking into what made the config disappear.

[bhargavdas]

Update: gpio controls validated by checking sysfs values of gpio48/value both using mtda-cli console and mtda-cli target on/off.

[chombourger]

CI fails on the following reuse errors:

The following files have no copyright and licensing information:
* kas/yocto/mtda-beaglebone-yocto.yml
* kas/yocto/mtda-qemu-yocto.yml
* meta-yocto/recipes-devtools/python/python3-daemon/0001-Workaround-for-issue-2-1.patch

[bhargavdas]

@chombourger @vj-kumar Please review the SRI_URI changes.

[bhargavdas]

Let's hold this PR for few more days, until python3-daemon recipe gets merged in meta-python

[bhargavdas]

Let's hold this PR for few more days, until python3-daemon recipe gets merged in meta-python

0f2c681
Updated the layer to support latest yocto release

[bhargavdas]

@chombourger , @vj-kumar this PR is ready to merge now.

[chombourger]

if you are wanting PR builds to check Yocto builds then this line should be removed

[bhargavdas]

Oh understood now, then we can keep this I guess, I have already checked the builds. Checking each PR build might take quite a lot of time.

[chombourger]

Any idea how long it would take? I am worried that Yocto support could easily get broken if not checked regularly
maybe we need to implement sstate caching like done in https://github.com/siemens/meta-coral ?

[bhargavdas]

more than 1hr in a 16 core machine. But I believe the machines provide by github actions are quite low powered.
We can only build the mtda package instead of building the whole image. Let me check in a similar machine for the same , will get a rough idea.

[bhargavdas]

I have added this stage (2f0a9f9) to build only the mtda package, please assign a check tag to check how much time it takes.
I triggered a mtda package build yesterday in a fairly older machine the build took around 4hrs.

[bhargavdas]

Looks like the space offered by the github ci machine are enough need find alternate methods :(

[bhargavdas]

@chombourger @vj-kumar how should we handle the ci for yocto since github's runners run out of space. Shall we run locally once a week ? or hook a self hosted runner? Suggestions.

[jluebbe]

@bhargavdas You should probably enable rm_work, which will at least help with the space usage issue.

[bhargavdas]

@chombourger , What should be the approach for this PR ? Since the runners provided by GH are not powerful enough shall I setup a build at my end locally and publish results weekly or monthly in some form here ?
Also looks like this PR needs rebase.