Most efficient way to loop through a Mat

[msmartin4470]

What is the best way to loop though a Mat in dart? In C I did this using pointers as follows. Thanks in advance.

for (int i = 0; i < mat.rows; i++)
{
	uint16_t* matRow = mat.ptr<uint16_t>(i);
	for (int j = 0; j < mat.cols; j++)
	{
		uint16_t value = matRow[j];
		matRow[j] = lut.at<uint16_t>(value);
	}
}

[rainyl]

UPDATE: Since v1.2.0, Mat.at, Mat.set, Mat.ptrAt, etc. had been improved significantly, refer to #193 for more info.

@msmartin4470 You can use Mat.at<T>() too, but only for getter, if you need setter, use Mat.set<T>

Although not tested, I think it is an efficient way to loop through a mat for now.

[msmartin4470]

It seems that using at() and set() takes quite a bit more time that my existing solution which it implemented in c++ can called through a method channel. I will work with the new ptr functions to see if it improves the speed.

[rainyl]

Your c++ implementation copies fewer times between dart side and c++ side, so it's unfair.

It is forseeable that at() and set() will be slow, I don't know your test code but it also surprised me, dart do not support functions overloading so if else is used to determine the type internally,

opencv_dart/lib/src/core/mat.dart

Lines 240 to 440 in 7e7a3df

	T _atNum<T>(int row, int col, [int? cn]) {
	return cvRunArena<T>((arena) {
	switch (type.depth) {
	case MatType.CV_8U:
	final p = arena<ffi.Uint8>();
	if (type.channels == 1 || cn == null) {
	cvRun(() => CFFI.Mat_GetUChar(ref, row, col, p));
	} else {
	cvRun(() => CFFI.Mat_GetUChar3(ref, row, col, cn, p));
	}
	return p.value as T;
	case MatType.CV_8S:
	final p = arena<ffi.Int8>();
	if (type.channels == 1 || cn == null) {
	cvRun(() => CFFI.Mat_GetSChar(ref, row, col, p));
	} else {
	cvRun(() => CFFI.Mat_GetSChar3(ref, row, col, cn, p));
	}
	return p.value as T;
	case MatType.CV_16U:
	final p = arena<ffi.Uint16>();
	if (type.channels == 1 || cn == null) {
	cvRun(() => CFFI.Mat_GetUShort(ref, row, col, p));
	} else {
	cvRun(() => CFFI.Mat_GetUShort3(ref, row, col, cn, p));
	}
	return p.value as T;
	case MatType.CV_16S:
	final p = arena<ffi.Int16>();
	if (type.channels == 1 || cn == null) {
	cvRun(() => CFFI.Mat_GetShort(ref, row, col, p));
	} else {
	cvRun(() => CFFI.Mat_GetShort3(ref, row, col, cn, p));
	}
	return p.value as T;
	case MatType.CV_32S:
	final p = arena<ffi.Int32>();
	if (type.channels == 1 || cn == null) {
	cvRun(() => CFFI.Mat_GetInt(ref, row, col, p));
	} else {
	cvRun(() => CFFI.Mat_GetInt3(ref, row, col, cn, p));
	}
	return p.value as T;
	case MatType.CV_32F:
	final p = arena<ffi.Float>();
	if (type.channels == 1 || cn == null) {
	cvRun(() => CFFI.Mat_GetFloat(ref, row, col, p));
	} else {
	cvRun(() => CFFI.Mat_GetFloat3(ref, row, col, cn, p));
	}
	return p.value as T;
	case MatType.CV_64F:
	final p = arena<ffi.Double>();
	if (type.channels == 1 || cn == null) {
	cvRun(() => CFFI.Mat_GetDouble(ref, row, col, p));
	} else {
	cvRun(() => CFFI.Mat_GetDouble3(ref, row, col, cn, p));
	}
	return p.value as T;
	default:
	throw UnsupportedError("at() for $type is not supported!");
	}
	});
	}
	T _atVec<T>(int row, int col) {
	final v = cvRunArena<T>((arena) {
	// Vec2b, Vec3b, Vec4b
	if (T == Vec2b) {
	final p = arena<cvg.Vec2b>();
	cvRun(() => CFFI.Mat_GetVec2b(ref, row, col, p));
	return Vec2b.fromNative(p.ref) as T;
	} else if (T == Vec3b) {
	final p = arena<cvg.Vec3b>();
	cvRun(() => CFFI.Mat_GetVec3b(ref, row, col, p));
	return Vec3b.fromNative(p.ref) as T;
	} else if (T == Vec4b) {
	final p = arena<cvg.Vec4b>();
	cvRun(() => CFFI.Mat_GetVec4b(ref, row, col, p));
	return Vec4b.fromNative(p.ref) as T;
	}
	// Vec2w, Vec3w, Vec4w
	else if (T == Vec2w) {
	final p = arena<cvg.Vec2w>();
	cvRun(() => CFFI.Mat_GetVec2w(ref, row, col, p));
	return Vec2w.fromNative(p.ref) as T;
	} else if (T == Vec3w) {
	final p = arena<cvg.Vec3w>();
	cvRun(() => CFFI.Mat_GetVec3w(ref, row, col, p));
	return Vec3w.fromNative(p.ref) as T;
	} else if (T == Vec4w) {
	final p = arena<cvg.Vec4w>();
	cvRun(() => CFFI.Mat_GetVec4w(ref, row, col, p));
	return Vec4w.fromNative(p.ref) as T;
	}
	// Vec2s, Vec3s, Vec4s
	else if (T == Vec2s) {
	final p = arena<cvg.Vec2s>();
	cvRun(() => CFFI.Mat_GetVec2s(ref, row, col, p));
	return Vec2s.fromNative(p.ref) as T;
	} else if (T == Vec3s) {
	final p = arena<cvg.Vec3s>();
	cvRun(() => CFFI.Mat_GetVec3s(ref, row, col, p));
	return Vec3s.fromNative(p.ref) as T;
	} else if (T == Vec4s) {
	final p = arena<cvg.Vec4s>();
	cvRun(() => CFFI.Mat_GetVec4s(ref, row, col, p));
	return Vec4s.fromNative(p.ref) as T;
	}
	// Vec2i, Vec3i, Vec4i, Vec6i, Vec8i
	else if (T == Vec2i) {
	final p = arena<cvg.Vec2i>();
	cvRun(() => CFFI.Mat_GetVec2i(ref, row, col, p));
	return Vec2i.fromNative(p.ref) as T;
	} else if (T == Vec3i) {
	final p = arena<cvg.Vec3i>();
	cvRun(() => CFFI.Mat_GetVec3i(ref, row, col, p));
	return Vec3i.fromNative(p.ref) as T;
	} else if (T == Vec4i) {
	final p = arena<cvg.Vec4i>();
	cvRun(() => CFFI.Mat_GetVec4i(ref, row, col, p));
	return Vec4i.fromNative(p.ref) as T;
	} else if (T == Vec6i) {
	final p = arena<cvg.Vec6i>();
	cvRun(() => CFFI.Mat_GetVec6i(ref, row, col, p));
	return Vec6i.fromNative(p.ref) as T;
	} else if (T == Vec8i) {
	final p = arena<cvg.Vec8i>();
	cvRun(() => CFFI.Mat_GetVec8i(ref, row, col, p));
	return Vec8i.fromNative(p.ref) as T;
	}
	// Vec2f, Vec3f, Vec4f, Vec6f
	else if (T == Vec2f) {
	final p = arena<cvg.Vec2f>();
	cvRun(() => CFFI.Mat_GetVec2f(ref, row, col, p));
	return Vec2f.fromNative(p.ref) as T;
	} else if (T == Vec3f) {
	final p = arena<cvg.Vec3f>();
	cvRun(() => CFFI.Mat_GetVec3f(ref, row, col, p));
	return Vec3f.fromNative(p.ref) as T;
	} else if (T == Vec4f) {
	final p = arena<cvg.Vec4f>();
	cvRun(() => CFFI.Mat_GetVec4f(ref, row, col, p));
	return Vec4f.fromNative(p.ref) as T;
	} else if (T == Vec6f) {
	final p = arena<cvg.Vec6f>();
	cvRun(() => CFFI.Mat_GetVec6f(ref, row, col, p));
	return Vec6f.fromNative(p.ref) as T;
	}
	// Vec2d, Vec3d, Vec4d, Vec6d
	else if (T == Vec2d) {
	final p = arena<cvg.Vec2d>();
	cvRun(() => CFFI.Mat_GetVec2d(ref, row, col, p));
	return Vec2d.fromNative(p.ref) as T;
	} else if (T == Vec3d) {
	final p = arena<cvg.Vec3d>();
	cvRun(() => CFFI.Mat_GetVec3d(ref, row, col, p));
	return Vec3d.fromNative(p.ref) as T;
	} else if (T == Vec4d) {
	final p = arena<cvg.Vec4d>();
	cvRun(() => CFFI.Mat_GetVec4d(ref, row, col, p));
	return Vec4d.fromNative(p.ref) as T;
	} else if (T == Vec6d) {
	final p = arena<cvg.Vec6d>();
	cvRun(() => CFFI.Mat_GetVec6d(ref, row, col, p));
	return Vec6d.fromNative(p.ref) as T;
	} else {
	throw UnsupportedError("at<$T>() for $type is not supported!");
	}
	});
	return v;
	}
	/// cv::Mat::at\<T\>(i0, i1, i2) of cv::Mat
	///
	///
	/// - If matrix is of type [MatType.CV_8U] then use Mat.at\<uchar\>(y,x).
	/// - If matrix is of type [MatType.CV_8S] then use Mat.at\<schar\>(y,x).
	/// - If matrix is of type [MatType.CV_16U] then use Mat.at\<ushort\>(y,x).
	/// - If matrix is of type [MatType.CV_16S] then use Mat.at\<short\>(y,x).
	/// - If matrix is of type [MatType.CV_32S] then use Mat.at\<int\>(y,x).
	/// - If matrix is of type [MatType.CV_32F] then use Mat.at\<float\>(y,x).
	/// - If matrix is of type [MatType.CV_64F] then use Mat.at\<double\>(y,x).
	///
	/// example:
	/// ```dart
	/// var m = cv.Mat.fromScalar(cv.Scalar(2, 4, 1, 0), cv.MatType.CV_32FC3);
	/// m.at<double>(0, 0); // 2
	/// m.at<cv.Vec3f>(0, 0); // cv.Vec3f(2, 4, 1)
	/// ```
	///
	/// https://docs.opencv.org/4.x/d3/d63/classcv_1_1Mat.html#a7a6d7e3696b8b19b9dfac3f209118c40
	T at<T>(int row, int col, [int? i2]) {
	if (T == int || T == double) {
	return _atNum<T>(row, col, i2);
	} else if (isSubtype<T, CvVec>()) {
	return _atVec<T>(row, col);
	} else {
	throw UnsupportedError("T must be num or CvVec(e.g., Vec3b), but got $T");
	}
	}

maybe we should just expose internal functions like atU8, atF64... atVec3b, etc., but it will be laborious...

I will test the overhead of type determination when having some time, and try to improve the performance if the overhead is unacceptable.

[rainyl]

@msmartin4470 Recently I have implemented the Mat::ptr in v1.0.3 and v2.0.0-dev.4 to allow accessing pointer like c++, with proper and careful operation this might be more efficient, you can try it and open new issues if there are any problems.

Mat::ptr<T>() -> Mat.ptrAt<T>()

or

Mat::ptr<uchar>() -> Mat.ptrU8()

and more similar APIs, you can explore more in release and document.

[msmartin4470]

Do you have any quick thoughts on how to speed this up? It is just applying a lookup table to a 16 bit image. It is taking about 1500ms currently, and it's the one piece of my project that I need up run as fast as possible. Thanks.

    final windowLeveled = src.clone();
    for (int r = 0; r < src.rows; r++) {
      final srcRow = src.ptrU16(r);
      final windowLeveledRow = windowLeveled.ptrU16(r);
      for (int c = 0; c < src.cols; c++) {
        final value = lut[srcRow[c]];
        windowLeveledRow[c] = value;
      }
    }

[rainyl]

https://docs.opencv.org/4.x/db/da5/tutorial_how_to_scan_images.html#:~:text=Goal

I think cv.LUT() is what you need, but it seems the api was not implemented in opencv_dart, if you need it urgently, you can try to add it and open a PR, or I will add it recently.

Actually it will be easy, the API exists in C wrappers but just not added to dart implementation, even no need to recompile the dynamic library.

[rainyl]

oh, sorry it seems LUT is only for CV_8U/S, so maybe we need to implement a more universal function like LUT

[msmartin4470]

Yes LUT is exactly what I need. As you said it does not support CV_16U, so I am a bit stuck. Thank you for looking into it.

[rainyl]

@msmartin4470 I have published v1.0.4 with cv.LUT supported, please upgrade and test if it works and works as expected.