Visual artifacts on image_rect ROS topic when using "bayer_rggb8" encoding

[davidkh1]

When ROS-Driver for Basler Cameras is configured to use bayer_bggr8 efficient encoding, visual artifacts appear in rectified images. Image stream on image_raw topic is OK in both cases.

What is the root cause of this issue?

Attached:

1. example of a problematic image from image_rect topic when in .yaml file
   image_encoding: "bayer_rggb8"
2. image is OK when image_encoding: "rgb8" in yaml file.

[schornakj]

I've run into this as well. In other CV applications a de-Bayering step is required before rectification to calculate the color values for each pixel from their compressed representation. I think the problem is that at line 521 in pylon_camera_node.cpp the raw image data is passed to rectifyImage without accounting for compression, which results in the loss of the compressed colors as the pixels get shifted around.

pylon_camera/src/pylon_camera/pylon_camera_node.cpp

Lines 513 to 523 in 2a7a068

	if ( getNumSubscribersRect() > 0 && camera_info_manager_->isCalibrated() )
	{
	cv_bridge_img_rect_->header.stamp = img_raw_msg_.header.stamp;
	assert(pinhole_model_->initialized());
	cv_bridge::CvImagePtr cv_img_raw = cv_bridge::toCvCopy(
	img_raw_msg_,
	img_raw_msg_.encoding);
	pinhole_model_->fromCameraInfo(camera_info_manager_->getCameraInfo());
	pinhole_model_->rectifyImage(cv_img_raw->image, cv_bridge_img_rect_->image);
	img_rect_pub_->publish(*cv_bridge_img_rect_);
	}

This might be tricky to fix while still keeping the node's image processing pipeline generic to image formats. One option would be to check the encoding at line 517 and convert to an uncompressed image format if needed.

[fho306]

I ran into this exact issue as well and found two ways to fix it:

A. Convert the image before rectification:
`cv::cvtColor(cv_img_raw->image, cv_img_debayer_->image, cv::COLOR_BayerBG"BGR);'
'cv_bridge_img_rect_->encoding = "rgb8";'

B. Publish the rectified image via image_transport instead of ros::Publisher*
See this pull request on the newer version of this driver:
basler/pylon-ros-camera#11
This is the better solution in my opinion, as it is still generic to image formats and I'd want to use image_transport for publishing images anyways.

Because of solution B I actually think the issue is that ros::Publisher* does not support the 'bayer_bggr8' encoding.