HDRLoader: Handle RGBA source Buffer

[ThomasGorisse]

Hi,

HDRLoader is doing a great job coupled to the SpecularFilter.

filament/android/filament-utils-android/src/main/cpp/HDRLoader.cpp

Lines 36 to 91 in 9ff195c

	jlong nCreateHDRTexture(JNIEnv* env, jclass,
	jlong nativeEngine, jobject javaBuffer, jint remaining, jint internalFormat) {
	Engine* engine = (Engine*) nativeEngine;
	AutoBuffer buffer(env, javaBuffer, remaining);
	Texture::InternalFormat textureFormat = (Texture::InternalFormat) internalFormat;
	auto dataPtr = (char const*) buffer.getData();
	const size_t byteCount = buffer.getSize();
	// This creates a copy but it's the easest way to create a memory stream.
	std::string ins(dataPtr, byteCount);
	std::istringstream in(ins);
	LinearImage* image = new LinearImage(ImageDecoder::decode(in, "memory.hdr"));
	// This can happen if a decoding error occurs.
	if (image->getChannels() != 3) {
	delete image;
	return 0;
	}
	Texture* texture = Texture::Builder()
	.width(image->getWidth())
	.height(image->getHeight())
	.levels(0xff)
	.sampler(Texture::Sampler::SAMPLER_2D)
	.format(textureFormat)
	.build(*engine);
	if (texture == nullptr) {
	slog.e << "Unable to create Filament Texture from HDR image." << io::endl;
	delete image;
	return 0;
	}
	PixelBufferDescriptor::Callback freeCallback = [](void* buf, size_t, void* userdata) {
	delete (LinearImage*) userdata;
	};
	PixelBufferDescriptor pbd(
	(void const* ) image->getPixelRef(),
	image->getWidth() * image->getHeight() * 3 * sizeof(float),
	PixelBufferDescriptor::PixelDataFormat::RGB,
	PixelBufferDescriptor::PixelDataType::FLOAT,
	freeCallback,
	image);
	// Note that the setImage call could fail (e.g. due to an invalid combination of internal format
	// and PixelDataFormat) but there is no way of detecting such a failure.
	texture->setImage(*engine, 0, std::move(pbd));
	texture->generateMipmaps(*engine);
	return (jlong) texture;
	}

But I have a case where my source buffer is in RGBA16F (ARCore estimated cubemap).
This involve that I have to create a new buffer in order to remove the alpha channel.
This wouldn't be a big issue in a normal usage case but in mine, I have to do it on every frame.

Would it possible to add an input format param to nCreateHDRTexture?

If it's not applicable do you have any idea of the best way/less consuming way to strip the alpha channel from the buffer ?

Thanks

[ThomasGorisse]

My alpha bytes skipping solution doesn't seems to work:

HDRLoader.createEnvironment(ByteBuffer.allocateDirect(
                arImages[0].width * arImages[0].height *
                        arImages.size *
                        // RGB Bytes per pixel
                        6 * 2
            ).apply {
                // Use the device hardware's native byte order
                order(ByteOrder.nativeOrder())
                val rgbaBytes = ByteArray(8) // ARGB Bytes per pixel
                arImages.forEach { image ->
                    image.planes[0].buffer.let { imageBuffer ->
                        while (imageBuffer.hasRemaining()) {
                            // Only take the RGB channels
                            put(rgbaBytes.apply {
                                imageBuffer.get(this)
                            } // Skip the Alpha channel
                                .sliceArray(0..5))
                        }
                    }
                }
            })

[ThomasGorisse]

Sorry,

I just discovered what an hdr file is and that it's not only a RGB container at all.
My mind just went from ARCore gives me 6 HDR images and filament can create a texture from an HDR file to let's create an HDR texture from the image content.

[romainguy]

Why do you want to skip the alpha channel? RGB16F is not a native format and drivers/GPUs (or in the case of Vulkan/Filament) have to expand it to RGBA16F anyway.

[ThomasGorisse]

Because of this on the AR Core part:
An array that contains 6 images in AIMAGE_FORMAT_RGBA_FP16 format for the HDR cubemap. The memory layout for the image data is identical to GL_RGBA16F

And this on the Filament part:

filament/android/filament-android/src/main/java/com/google/android/filament/Texture.java

Line 1097 in 9ff195c

* <li>the data {@link Format format} must be {@link Format#RGB}</li>