Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

minimal required mt9m114 driver changes #6

Open
rmast opened this issue Oct 18, 2024 · 27 comments
Open

minimal required mt9m114 driver changes #6

rmast opened this issue Oct 18, 2024 · 27 comments

Comments

@rmast
Copy link

rmast commented Oct 18, 2024 

          > As I expect the rewritten Onsemi driver to abide closer to the new v4l2 I'm curious what would happen if I add { "INT33F0" } to the of_device_id call. It has a slightly different name from the acpi_device_id call in the old driver.

@jwrdegoede writes:
Unfortunately things are not that easy, I have a pretty good idea what the minimum necessary changes are, to get things to work at a minimum the following mt9m114 driver changes are required:

  1. Add the ACPI and support for ACPI based enumeration in general which may require further changes.
  2. Waiting for the fwnode endpoint to show up before continuing with probing
  3. Special handling for the clk, devm_get_clk() is expected to fail since the clk is managed by ACPI
  4. Maybe handling of a second (powerdown) GPIO
  5. atomisp changes to deal with the standard v4l2 driver offering multiple mbus formats instead of only offering a single raw bayer format.

Originally posted by @jwrdegoede in #4 (comment)
@rmast
Copy link
Author

rmast commented Oct 18, 2024

@jwrdegoede

  1. As ACPI based enumeration seemed to have long been a no go area, is that something that requires active Intel support, or is another strategy feasible? I guess this one is only meant to be able to skip the hardly reachable PCI enumeration?
  2. I guess GPT-4o can make sense of this.
  3. Can we just skip the clock handling in this case, or is a replacing ACPI clk handler required? Can we write a GPT-4o prompt to make this happen?
  4. and 5. also seem not really minimal to get a first picture out of the new driver.

Do we additionally have to transplant some of the GPIO handling of the atomisp-mt9m114.c to some atomisp or v4l2-driver to make the new mt9m114.c work?

@rmast
Copy link
Author

rmast commented Oct 21, 2024

The devil is in the detail:

mt9m114.c:
static struct i2c_driver mt9m114_driver = {
.driver = {
.name = "mt9m114",
.pm = &mt9m114_pm_ops,
.of_match_table = mt9m114_of_ids,
},
.probe = mt9m114_probe,
.remove = mt9m114_remove,
};

ov2680.c:

static struct i2c_driver ov2680_i2c_driver = {
.driver = {
.name = "ov2680",
.pm = pm_sleep_ptr(&ov2680_pm_ops),
.of_match_table = ov2680_dt_ids,
.acpi_match_table = ov2680_acpi_ids,
},
.probe = ov2680_probe,
.remove = ov2680_remove,
};

@rmast
Copy link
Author

rmast commented Oct 23, 2024

This source:
https://github.com/torvalds/linux/blob/master/drivers/staging/media/atomisp/pci/atomisp_csi2_bridge.c

should be adapted for Baytrail as was suggested in this commit message:

https://git.kernel.org/pub/scm/linux/kernel/git/hansg/linux.git/commit/drivers/staging/media/atomisp/i2c/atomisp-ov2680.c?h=media-atomisp&id=b7e155e16601b0c6f34e3345b9eca6a2efc5bc5d

Currently this source only supports Lenovo Ideapad Miix 310 and Xiaomi Mipad2, which both depend on a well formed Cherry Trail GUID for the camera in the DSDT, where the DSDT of the Asus T100 is not that mature and doesn't contain this GUID:
dc2f6c4f-045b-4f1d-97b9-882a6860a4be

@rmast
Copy link
Author

rmast commented Oct 23, 2024

And probably this one as well: https://github.com/torvalds/linux/blob/master/drivers/media/pci/intel/ipu-bridge.c

@rmast
Copy link
Author

rmast commented Oct 23, 2024

This file has a comment that only mentions the atomisp2401:
https://github.com/torvalds/linux/blob/master/drivers/staging/media/atomisp/pci/atomisp_csi2.c

@rmast
Copy link
Author

rmast commented Oct 24, 2024

This function is a model for how parse_dt should await the bridge:
https://github.com/torvalds/linux/blob/master/drivers/media/i2c/ov2680.c#L1074

@rmast
Copy link
Author

rmast commented Oct 27, 2024

You could wonder why https://github.com/torvalds/linux/blob/master/drivers/media/i2c/aptina-pll.c is not applied in mt9m114.c for determining the PLL values.

@rmast
Copy link
Author

rmast commented Nov 20, 2024

I now pulled and built the newest media-atomisp-branch from Hansg:
https://git.kernel.org/pub/scm/linux/kernel/git/hansg/linux.git/log/?h=media-atomisp

I found this instruction for coping with memory-issues during driver building:
https://stackoverflow.com/questions/55278268/gstreamer-v4l2src-failed-to-allocate-required-memory
After installing and running v4l2-compliance I have the following status about V4L2_MEMORY_MMAP of the atomisp-mt9m114-driver:

v4l2-compliance 1.28.1, 64 bits, 64-bit time_t

Compliance test for atomisp device /dev/video0:

Driver Info:
	Driver name      : atomisp
	Card type        : ATOM ISP
	Bus info         : PCI:0000:00:03.0
	Driver version   : 6.12.0
	Capabilities     : 0x84200001
		Video Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04200001
		Video Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : atomisp-isp2
	Model            : Intel Atom ISP
	Serial           : 
	Bus info         : PCI:0000:00:03.0
	Media version    : 6.12.0
	Hardware revision: 0x00001010 (4112)
	Driver version   : 6.12.0
Interface Info:
	ID               : 0x03000019
	Type             : V4L Video
Entity Info:
	ID               : 0x00000017 (23)
	Name             : ATOMISP video output
	Function         : V4L2 I/O
	Pad 0x01000018   : 0: Sink
	  Link 0x0200001b: from remote pad 0x100000e of entity 'Atom ISP' (Image Signal Processor): Data, Enabled, Immutable

Required ioctls:
	test MC information (see 'Media Driver Info' above): OK
	test VIDIOC_QUERYCAP: OK
		fail: v4l2-compliance.cpp(810): doioctl(node, ioc, nullptr) != ENOTTY
	test invalid ioctls: FAIL

Allow for multiple opens:
	test second /dev/video0 open: FAIL
		fail: v4l2-compliance.cpp(763): !ok
	test for unlimited opens: FAIL

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 1 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls (Input 0):
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 0 Private Controls: 0

Format ioctls (Input 0):
		fail: v4l2-test-formats.cpp(368): Accepted framesize for invalid format
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: FAIL
		fail: v4l2-test-formats.cpp(1331): cap->capturemode & ~V4L2_MODE_HIGHQUALITY
	test VIDIOC_G/S_PARM: FAIL
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK (Not Supported)
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls (Input 0):
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls (Input 0):
		fail: v4l2-test-buffers.cpp(672): node->node2 == nullptr
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
	test CREATE_BUFS maximum buffers: OK
	test VIDIOC_REMOVE_BUFS: OK
		fail: v4l2-test-buffers.cpp(1006): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
		fail: v4l2-test-buffers.cpp(1007): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
	test VIDIOC_EXPBUF: OK (Not Supported)
	test Requests: OK (Not Supported)

Total for atomisp device /dev/video0: 46, Succeeded: 40, Failed: 6, Warnings: 0

@rmast
Copy link
Author

rmast commented Nov 20, 2024

ChatGPT proposes to implement VIDEOBUF2-stuff for supporting V4L2_MEMORY_MMAP

@jwrdegoede
Copy link

@rmast, I see that you are spending a lot of time on this. But it is unclear to me what exactly you are trying to achieve?

If you take my latest atomisp branch and build a kernel with that and then run:

gst-launch-1.0 v4l2src ! videoconvert ! xvimagesink

as described in:

drivers/staging/media/atomisp/TODO

then the camera should just work except that you will have bad exposure/gain values.

The next steps would be to either expose proper exposure/gain v4l2-controls from the atomisp specific mt9m114 driver; or make things work with the non staging standard v4l2 mt9m114 driver from drivers/media/i2c/ .

I have a bunch of patching working towards the latter here:

https://github.com/jwrdegoede/linux-sunxi/commits/main

As a first step I would suggest you to reproduce my working setup where "gst-launch-1.0 v4l2src ! videoconvert ! xvimagesink" works using my media-atomisp branch as base.

Then lets talk about what exactly you are trying to achieve / what your goal here is and see how we can reach that.

@rmast
Copy link
Author

rmast commented Nov 20, 2024

@rmast, I see that you are spending a lot of time on this. But it is unclear to me what exactly you are trying to achieve?
... the camera should just work except that you will have bad exposure/gain values.

Great! That's exactly what I'm trying to accomplish. Can you share your magic .config for building the right kernel options as well?

Or is there a simple make _some_config command that selects the right config from scratch?

@jwrdegoede
Copy link

Great! That's exactly what I'm trying to accomplish. Can you share your magic .config for building the right kernel options as well?

The config which I use for all my testing can be found here: https://github.com/jwrdegoede/linux-sunxi/blob/main/.config

This is Fedora's default kernel config with some local modifications like enabling the atomisp driver.

Together with my atomisp branch from: https://git.kernel.org/pub/scm/linux/kernel/git/hansg/linux.git/log/?h=media-atomisp

This should result in a working camera. Assuming that you have done the BIOS hack to turn the atomisp device into a PCI device.

Note that with these old-style atomisp drivers it is important that the sensor driver gets loaded before the atomisp driver itself.

I blacklist the atomisp module and then manually modprobe it after atomisp-mt9m114 has loaded.

@rmast
Copy link
Author

rmast commented Nov 20, 2024

Still no luck and no /dev/video0.

This is my /etc/modprobe.d/atomisp.conf:
atomisp.conf.txt
lsmod after sudo modprobe atomisp:
lsmod.txt
journalctl -k:
journalctl-k.txt

Nov 20 16:51:43 fedora kernel: atomisp-isp2 0000:00:03.0: probe with driver atomisp-isp2 failed with error -22

@rmast
Copy link
Author

rmast commented Nov 21, 2024

@jwrdegoede as this commit torvalds/linux@795ac29#diff-e28901ef99c4094d9194d0bb3bd3ae194aead0287183eac5c81290c92b8b6479R62 contains a hacky solution to a similar problem, do you remember altering that video format manually and temporarily to get the mt9m114 to work?

@rmast
Copy link
Author

rmast commented Nov 21, 2024

And if so, did you pick similar values to the allowed values for the mt9m114 on this page: https://github.com/jfwells/linux-asus-t100ta/tree/master/webcam

for example
V4L2_PIX_FMT_YUV422P or V4L2_PIX_FMT_RGB565

@rmast
Copy link
Author

rmast commented Nov 22, 2024

Yes yes yes yes yessss!!!!

V4L2_PIX_FMT_YUV422P did the trick!

Next step will be to provide this result in a dkms-like construct with @EasyNetDev to make it possible to promote and continue development without rebuilding the kernel each time.

@jotamanso
Copy link

jotamanso commented Nov 22, 2024 via email

@rmast rmast mentioned this issue Nov 22, 2024
Open
@rmast
Copy link
Author

rmast commented Nov 22, 2024

habemus camera?

prima signa

@jotamanso
Copy link

jotamanso commented Nov 22, 2024 via email

@rmast
Copy link
Author

rmast commented Nov 22, 2024 via email •
edited
Loading

@rmast
Copy link
Author

rmast commented Nov 22, 2024 via email •
edited
Loading

@rmast
Copy link
Author

rmast commented Nov 23, 2024 via email •
edited
Loading

@rmast
Copy link
Author

rmast commented Nov 24, 2024 via email •
edited
Loading

@rmast
Copy link
Author

rmast commented Dec 8, 2024 via email

@rmast
Copy link
Author

rmast commented Dec 8, 2024 via email

@EasyNetDev
Copy link

EasyNetDev commented Dec 8, 2024 •
edited
Loading

This bug is not related to Baytrail CPU bug that hangs the OS in the past?

@rmast
Copy link
Author

rmast commented Dec 9, 2024 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@jwrdegoede @EasyNetDev @rmast @jotamanso