lmdb-to-hdfs

Small utility to load LMDB datastore into HDFS as a sequence file.

Pre-configured to run on Linux 64-bit. To run on other architectures (Windows, OSX, Android), change the Maven dependency in this line within pom.xml. Possible values are:

• lmdbjni-linux64
• lmdbjni-win64
• lmdbjni-osx64
• lmdbjni-android

See the lmdbjni project home page for more information.

The Maven page for this library is here.

Usage

java -jar lmdbToHdfs.jar <lmdb-path> <hdfs-path> [<resource-uri>..]

• lmdb-path is a directory containing an LMDB database, e.g. /var/imagenet/lmdb
• hdfs-path is a path for (new) sequence file on HDFS, e.g. hdfs://master:9000/imagenet/lmdb
• resource-uri (0 or more values) are URIs of Hadoop configuration file, e.g. file:///usr/local/hadoop/etc/hadoop/core-site.xml

NOTE: When not specifying any Hadoop configuration files, Hadoop's default Configuration will be used