diff --git a/docs/getting_started.md b/docs/getting_started.md index 661110fa..7a676a12 100644 --- a/docs/getting_started.md +++ b/docs/getting_started.md @@ -1,8 +1,10 @@ # Getting started with Deep-Image-Matching -Deep-Image-Matching can be launched from the Command Line (CLI), from the GUI (note that the GUI is still under development) or use Deep_Image_Matching as a Python library. +Deep-Image-Matching can be launched from the Command Line (CLI), from the GUI (note that the GUI is still under development) or use Deep_Image_Matching as a Python library. -## Command Line Interface (CLI) +## Run Deep-Image-Matching + +### Command Line Interface (CLI) Before running the CLI, check the options with `python ./main.py --help`. @@ -27,12 +29,11 @@ Other optional parameters are: Finally, there are some 'strategy-dependent' options (i.e., options that are used only with specific strategies). See [Matching strategies](#matching-strategies) section for more information. These options are: -- `--overlap`: if 'strategy' is set to 'sequential', set the number of images that are sequentially matched in the sequence (default: `1`) +- `--overlap`: if 'strategy' is set to 'sequential', set the number of images that are sequentially matched in the sequence (default: `1`) - `--global_feature`: if `strategy` is set to `retrieval`, set the global descriptor to use for image retrieval. Options are: "netvlad", "openibl", "cosplace", "dir" (default: `netvlad`). - `--pair_file`: if `strategy` is set to `custom_pairs`, set the path to the text file containing the pairs of images to be matched. (default: `None`). - -## Graphical User Interface (GUI) +### Graphical User Interface (GUI) **Note that the GUI is still under development and it may have some bugs** @@ -45,15 +46,15 @@ python ./main.py --gui In the GUI, you can define the same parameters that are available in the CLI. The GUI loads the available configurations from [`config.py`](https://github.com/3DOM-FBK/deep-image-matching/blob/master/src/deep_image_matching/config.py) located in `/src/deep_image_matching`. -## From a Jupyter notebooks +### From Jupyter notebooks If you want to use Deep_Image_Matching from a Jupyter notebook, you can check the examples in the [`notebooks`](https://github.com/3DOM-FBK/deep-image-matching/tree/master/notebooks) folder. ## Basic configuration -### Pipeline +### Pipelines -The `pipeline` defines the combination of local feature extractor and matcher to be used for the matching is is defined by the `--pipeline` option in the CLI. +The `pipeline` parameter defines the combination of local feature extractor and matcher to be used for the matching is is defined by the `--pipeline` option in the CLI. Possible configurations are: @@ -93,11 +94,11 @@ More information can be obtained looking to the code in [`config.py`](https://gi The matching strategy defines how the pairs of images to be matches are selected. Available matching strategies are: - `matching_lowres`: the images are first matched at low resolution (resizing images with the longest edge to 1000 px) and candidates are selected based on the number of matches (the minimum number of matches is 20). Once the candidate pairs are selected, the images are matched at the desired resolution, specified by the `Quality` parameter in the configuration. This is the default option and the recommended strategy, especially for large datasets. -- `bruteforce`: all the possible pairs of images are matched. This is is useful in case of very challenging datasets, where some image pairs may be rejected by the previous strategies, but it can take significantly more time for large datasets. +- `bruteforce`: all the possible pairs of images are matched. This is is useful in case of very challenging datasets, where some image pairs may be rejected by the previous strategies, but it can take significantly more time for large datasets. - `sequential`: the images are matched sequentially, i.e., the first image is matched with the second, the second with the third, and so on. The number of images to be matched sequentially is defined by the `--overlap` option in the CLI. This strategy is useful for datasets where the images are taken sequentially, e.g., from a drone or a car. - `retrieval`: the images are first matched based with a global descriptor to select the pairs. The global descriptor to be used is defined by the `--retrieval` option in the CLI. Available global descriptors are: "netvlad", "openibl", "cosplace", "dir". -- `custom_pairs`: the pairs of images to be matched are defined in a text file. The path to the text file is defined by the `--pairs` option in the CLI. The text file must contain one pair of images per line, separated by a space. The images must be identified by their full name (i.e., the name of the image file with the extension). -For example: +- `custom_pairs`: the pairs of images to be matched are defined in a text file. The path to the text file is defined by the `--pairs` option in the CLI. The text file must contain one pair of images per line, separated by a space. The images must be identified by their full name (i.e., the name of the image file with the extension). + For example: ```text image_1.jpg image_2.jpg @@ -108,18 +109,16 @@ For example: The `Quality` parameter define the resolution at which the images are matched. The available options are: -- `highest`: each image size is upsampled by a factor of 2 by using a bicubic interpolation. -- `high`: the images are matched at the original resolution (default) -- `medium`: each image size is downsampled by a factor of 2 by using the OpenCV pixel-area approach. -- `low`: each image size is downsampled by a factor of 4 by using the OpenCV pixel-area approach. -- `lowest`: each image size is downsampled by a factor of 8 by using the OpenCV pixel-area approach. - -OpenCV pixel-area approach ([cv2.INTER_AREA](https://docs.opencv.org/4.x/da/d54/group__imgproc__transform.html#gga5bb5a1fea74ea38e1a5445ca803ff121acf959dca2480cc694ca016b81b442ceb)). +- `highest`: each image size is upsampled by a factor of 2 by using a bicubic interpolation ([cv2.INTER_CUBIC](https://docs.opencv.org/4.x/da/d54/group__imgproc__transform.html#gga5bb5a1fea74ea38e1a5445ca803ff121a55e404e7fa9684af79fe9827f36a5dc1)). +- `high`: images are matched at the original resolution (default) +- `medium`: images are downsampled by a factor of 2 by using the OpenCV pixel-area approach ([cv2.INTER_AREA](https://docs.opencv.org/4.x/da/d54/group__imgproc__transform.html#gga5bb5a1fea74ea38e1a5445ca803ff121acf959dca2480cc694ca016b81b442ceb)). +- `low`: images are downsampled by a factor of 4 by using the OpenCV pixel-area approach ([cv2.INTER_AREA](https://docs.opencv.org/4.x/da/d54/group__imgproc__transform.html#gga5bb5a1fea74ea38e1a5445ca803ff121acf959dca2480cc694ca016b81b442ceb)). +- `lowest`: images are downsampled by a factor of 8 by using the OpenCV pixel-area approach ([cv2.INTER_AREA](https://docs.opencv.org/4.x/da/d54/group__imgproc__transform.html#gga5bb5a1fea74ea38e1a5445ca803ff121acf959dca2480cc694ca016b81b442ceb)). ### Tiling If images have a high resolution (e.g., larger than 3000 px, but this limit depends on the memory of your GPU) and you do not want to downsample them (e.g., to avoid loosing accuracy in feature detection), it may be useful to carry out the matching by dividing the images into regular tiles. -This can be done by specifying the tiling approach with the `--tiling` option in the CLI. +This can be done by specifying the tiling approach with the `--tiling` option in the CLI. If you want to run the matching by tile, you can choose different approaches for selecting the tiles to be matched. Available options are: - `None`: no tiling is applied (default) @@ -129,7 +128,6 @@ If you want to run the matching by tile, you can choose different approaches for To control the tile size and the tile overlap, refer to the [Advanced configuration](#advanced-configuration) section. - ## Advanced configuration If you want to set any additional parameter, you can do it by editing the `config.yaml` file that must be located in the same directory of the `main.py` file. @@ -157,13 +155,12 @@ general: tile_overlap: 20 ``` -The `extractor` and `matcher` sections contain the parameters that control the local feature extractor and the matcher selected by the '--pipeline' option from the CLI (or from GUI). +The `extractor` and `matcher` sections contain the parameters that control the local feature extractor and the matcher selected by the '--pipeline' option from the CLI (or from GUI). Both the sections **must contain the name** of the local feature extractor or the matcher that will be used for the matching (the name must be the same as the one used in the `--pipeline` option in the CLI). -In addition, you can specify any other parameters for controlling the extractor and the matcher. -The default values of all the configuration parameters are defined in the [`config.py`](https://github.com/3DOM-FBK/deep-image-matching/blob/master/src/deep_image_matching/config.py) file located in `/src/deep_image_matching` directory. +In addition, you can specify any other parameters for controlling the extractor and the matcher. +The default values of all the configuration parameters are defined in the [`config.py`](https://github.com/3DOM-FBK/deep-image-matching/blob/master/src/deep_image_matching/config.py) file located in `/src/deep_image_matching` directory. Please, note that different extractors or matchers may have different parameters, so you need to check carefully the available parameters for each extractor/matcher in the file [`config.py`](https://github.com/3DOM-FBK/deep-image-matching/blob/master/src/deep_image_matching/config.py). - ```yaml extractor: name: "superpoint" diff --git a/main.py b/main.py index 224bef00..ac3c117d 100644 --- a/main.py +++ b/main.py @@ -1,7 +1,8 @@ import os import subprocess -from pathlib import Path from importlib import import_module +from pathlib import Path + from deep_image_matching import logger, timer from deep_image_matching.config import Config from deep_image_matching.image_matching import ImageMatching @@ -81,6 +82,7 @@ # Export in openMVG format if config.general["openmvg_conf"]: import yaml + with open(config.general["openmvg_conf"], "r") as file: openmvgcfg = yaml.safe_load(file) system_OS = openmvgcfg["general"]["OS"] @@ -107,7 +109,7 @@ if not os.path.exists(openmvg_reconstruction_dir): os.mkdir(openmvg_reconstruction_dir) logger.debug("OpenMVG Sequential/Incremental reconstruction") - + if system_OS == "windows": pRecons = subprocess.Popen( [ diff --git a/notebooks/sfm_pipeline.ipynb b/notebooks/sfm_pipeline.ipynb index 4019ec74..ed4d5ef8 100644 --- a/notebooks/sfm_pipeline.ipynb +++ b/notebooks/sfm_pipeline.ipynb @@ -2,7 +2,7 @@ "cells": [ { "cell_type": "code", - "execution_count": 5, + "execution_count": 2, "metadata": {}, "outputs": [], "source": [ @@ -31,7 +31,7 @@ }, { "cell_type": "code", - "execution_count": 6, + "execution_count": 3, "metadata": {}, "outputs": [ { @@ -84,14 +84,14 @@ }, { "cell_type": "code", - "execution_count": 7, + "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "\u001b[1;33m2024-01-19 18:27:15 | [WARNING ] ../assets/example_cyprus/results_superpoint+lightglue_matching_lowres_quality_high already exists, but the '--force' option is used. Deleting the folder.\u001b[0m\n" + "\u001b[1;33m2024-02-21 12:40:00 | [WARNING ] ../assets/example_cyprus/results_superpoint+lightglue_matching_lowres_quality_high already exists, but the '--force' option is used. Deleting the folder.\u001b[0m\n" ] } ], @@ -104,6 +104,7 @@ " \"tiling\": \"preselection\",\n", " \"skip_reconstruction\": False,\n", " \"force\": True,\n", + " \"openmvg\": None,\n", "}\n", "config = Config(cli_params)" ] @@ -117,7 +118,7 @@ }, { "cell_type": "code", - "execution_count": 8, + "execution_count": 6, "metadata": {}, "outputs": [ { @@ -127,6 +128,7 @@ "Config general:\n", "{'db_path': None,\n", " 'geom_verification': ,\n", + " 'graph': True,\n", " 'gv_confidence': 0.99999,\n", " 'gv_threshold': 4,\n", " 'image_dir': PosixPath('../assets/example_cyprus/images'),\n", @@ -134,6 +136,7 @@ " 'min_inlier_ratio_per_pair': 0.25,\n", " 'min_inliers_per_pair': 15,\n", " 'min_matches_per_tile': 10,\n", + " 'openmvg_conf': None,\n", " 'output_dir': PosixPath('../assets/example_cyprus/results_superpoint+lightglue_matching_lowres_quality_high'),\n", " 'overlap': None,\n", " 'pair_file': PosixPath('../assets/example_cyprus/results_superpoint+lightglue_matching_lowres_quality_high/pairs.txt'),\n", @@ -181,7 +184,7 @@ }, { "cell_type": "code", - "execution_count": 9, + "execution_count": 7, "metadata": {}, "outputs": [], "source": [ @@ -208,7 +211,7 @@ }, { "cell_type": "code", - "execution_count": 10, + "execution_count": 8, "metadata": {}, "outputs": [], "source": [ @@ -229,7 +232,7 @@ }, { "cell_type": "code", - "execution_count": 11, + "execution_count": 9, "metadata": {}, "outputs": [ { @@ -238,17 +241,16 @@ "text": [ "Loaded SuperPoint model\n", "Loaded SuperPoint model\n", - "\u001b[0;37m2024-01-19 18:27:16 | [INFO ] Running image matching with the following configuration:\u001b[0m\n", - "\u001b[0;37m2024-01-19 18:27:16 | [INFO ] Image folder: ../assets/example_cyprus/images\u001b[0m\n", - "\u001b[0;37m2024-01-19 18:27:16 | [INFO ] Output folder: ../assets/example_cyprus/results_superpoint+lightglue_matching_lowres_quality_high\u001b[0m\n", - "\u001b[0;37m2024-01-19 18:27:16 | [INFO ] Number of images: 10\u001b[0m\n", - "\u001b[0;37m2024-01-19 18:27:16 | [INFO ] Matching strategy: matching_lowres\u001b[0m\n", - "\u001b[0;37m2024-01-19 18:27:16 | [INFO ] Image quality: Quality.HIGH\u001b[0m\n", - "\u001b[0;37m2024-01-19 18:27:16 | [INFO ] Tile selection: TileSelection.PRESELECTION\u001b[0m\n", - "\u001b[0;37m2024-01-19 18:27:16 | [INFO ] Retrieval option: None\u001b[0m\n", - "\u001b[0;37m2024-01-19 18:27:16 | [INFO ] Overlap: None\u001b[0m\n", - "\u001b[0;37m2024-01-19 18:27:16 | [INFO ] Feature extraction method: superpoint\u001b[0m\n", - "\u001b[0;37m2024-01-19 18:27:16 | [INFO ] Matching method: lightglue\u001b[0m\n" + "\u001b[0;37m2024-02-21 12:40:10 | [INFO ] Running image matching with the following configuration:\u001b[0m\n", + "\u001b[0;37m2024-02-21 12:40:10 | [INFO ] Image folder: ../assets/example_cyprus/images\u001b[0m\n", + "\u001b[0;37m2024-02-21 12:40:10 | [INFO ] Output folder: ../assets/example_cyprus/results_superpoint+lightglue_matching_lowres_quality_high\u001b[0m\n", + "\u001b[0;37m2024-02-21 12:40:10 | [INFO ] Number of images: 10\u001b[0m\n", + "\u001b[0;37m2024-02-21 12:40:10 | [INFO ] Matching strategy: matching_lowres\u001b[0m\n", + "\u001b[0;37m2024-02-21 12:40:10 | [INFO ] Image quality: Quality.HIGH\u001b[0m\n", + "\u001b[0;37m2024-02-21 12:40:10 | [INFO ] Tile selection: TileSelection.PRESELECTION\u001b[0m\n", + "\u001b[0;37m2024-02-21 12:40:10 | [INFO ] Feature extraction method: superpoint\u001b[0m\n", + "\u001b[0;37m2024-02-21 12:40:10 | [INFO ] Matching method: lightglue\u001b[0m\n", + "\u001b[0;37m2024-02-21 12:40:10 | [INFO ] CUDA available: True\u001b[0m\n" ] } ], @@ -276,30 +278,30 @@ }, { "cell_type": "code", - "execution_count": 12, + "execution_count": 10, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "\u001b[0;37m2024-01-19 18:27:16 | [INFO ] Low resolution matching, generating pairs ..\u001b[0m\n", + "\u001b[0;37m2024-02-21 12:40:13 | [INFO ] Low resolution matching, generating pairs ..\u001b[0m\n", "Loaded SuperPoint model\n", - "\u001b[0;37m2024-01-19 18:27:16 | [INFO ] Extracting features from downsampled images...\u001b[0m\n" + "\u001b[0;37m2024-02-21 12:40:13 | [INFO ] Extracting features from downsampled images...\u001b[0m\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ - "100%|██████████| 10/10 [00:00<00:00, 12.94it/s]" + "100%|██████████| 10/10 [00:02<00:00, 4.27it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ - "\u001b[0;37m2024-01-19 18:27:17 | [INFO ] Matching downsampled images...\u001b[0m\n" + "\u001b[0;37m2024-02-21 12:40:15 | [INFO ] Matching downsampled images...\u001b[0m\n" ] }, { @@ -307,14 +309,14 @@ "output_type": "stream", "text": [ "\n", - "100%|██████████| 45/45 [00:01<00:00, 38.85it/s]" + "100%|██████████| 45/45 [00:00<00:00, 51.84it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ - "\u001b[0;37m2024-01-19 18:27:18 | [INFO ] Found 28 pairs.\u001b[0m\n" + "\u001b[0;37m2024-02-21 12:40:16 | [INFO ] Found 28 pairs.\u001b[0m\n" ] }, { @@ -339,7 +341,7 @@ }, { "cell_type": "code", - "execution_count": 13, + "execution_count": 11, "metadata": {}, "outputs": [], "source": [ @@ -357,15 +359,15 @@ }, { "cell_type": "code", - "execution_count": 14, + "execution_count": 12, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "\u001b[0;37m2024-01-19 18:27:18 | [INFO ] Extracting features with superpoint...\u001b[0m\n", - "\u001b[0;37m2024-01-19 18:27:18 | [INFO ] superpoint configuration: \u001b[0m\n", + "\u001b[0;37m2024-02-21 12:44:46 | [INFO ] Extracting features with superpoint...\u001b[0m\n", + "\u001b[0;37m2024-02-21 12:44:46 | [INFO ] superpoint configuration: \u001b[0m\n", "{'keypoint_threshold': 0.0005,\n", " 'max_keypoints': 8000,\n", " 'name': 'superpoint',\n", @@ -376,14 +378,14 @@ "name": "stderr", "output_type": "stream", "text": [ - "100%|██████████| 10/10 [00:03<00:00, 3.20it/s]" + "100%|██████████| 10/10 [00:03<00:00, 3.00it/s]" ] }, { "name": "stdout", "output_type": "stream", "text": [ - "\u001b[0;37m2024-01-19 18:27:21 | [INFO ] Features extracted!\u001b[0m\n" + "\u001b[0;37m2024-02-21 12:44:50 | [INFO ] Features extracted!\u001b[0m\n" ] }, { @@ -408,15 +410,15 @@ }, { "cell_type": "code", - "execution_count": 15, + "execution_count": 13, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "\u001b[0;37m2024-01-19 18:27:21 | [INFO ] Matching features with lightglue...\u001b[0m\n", - "\u001b[0;37m2024-01-19 18:27:21 | [INFO ] lightglue configuration: \u001b[0m\n", + "\u001b[0;37m2024-02-21 12:44:52 | [INFO ] Matching features with lightglue...\u001b[0m\n", + "\u001b[0;37m2024-02-21 12:44:52 | [INFO ] lightglue configuration: \u001b[0m\n", "{'depth_confidence': 0.95,\n", " 'filter_threshold': 0.1,\n", " 'flash': True,\n", @@ -424,15 +426,15 @@ " 'n_layers': 9,\n", " 'name': 'lightglue',\n", " 'width_confidence': 0.99}\n", - "\u001b[0;37m2024-01-19 18:27:21 | [INFO ] Matching features...\u001b[0m\n", - "\u001b[0;37m2024-01-19 18:27:21 | [INFO ] \u001b[0m\n" + "\u001b[0;37m2024-02-21 12:44:52 | [INFO ] Matching features...\u001b[0m\n", + "\u001b[0;37m2024-02-21 12:44:52 | [INFO ] \u001b[0m\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ - "100%|██████████| 28/28 [00:22<00:00, 1.22it/s]\n" + "100%|██████████| 28/28 [00:19<00:00, 1.42it/s]\n" ] } ], @@ -1369,7 +1371,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.10.12" + "version": "3.10.13" } }, "nbformat": 4, diff --git a/src/deep_image_matching/config.py b/src/deep_image_matching/config.py index 8a19f566..2bbe10cf 100644 --- a/src/deep_image_matching/config.py +++ b/src/deep_image_matching/config.py @@ -500,6 +500,11 @@ def parse_general_config(input_args: dict) -> dict: if args["verbose"]: change_logger_level(logger.name, "debug") + if args["openmvg"] is not None: + args["openmvg"] = Path(args["openmvg"]) + if not args["openmvg"].exists(): + raise ValueError(f"File {args['openmvg']} does not exist") + # Build configuration dictionary cfg = { "image_dir": args["images"], diff --git a/src/deep_image_matching/parser.py b/src/deep_image_matching/parser.py index b6cdf0a3..2b929cd9 100644 --- a/src/deep_image_matching/parser.py +++ b/src/deep_image_matching/parser.py @@ -135,6 +135,7 @@ def parse_cli() -> dict: parser.add_argument( "--openmvg", help="Path to openmvg config file'", + default=None, ) args = parser.parse_args() diff --git a/tests/test_config.py b/tests/test_config.py index 2a8c284b..8c83796b 100644 --- a/tests/test_config.py +++ b/tests/test_config.py @@ -59,11 +59,13 @@ def test_valid_basic_arguments(data_dir): "min_inlier_ratio_per_pair": 0.25, "try_match_full_images": False, } - assert all( - key in config.general and config.general[key] == expected_general[key] - for key in expected_general - ) + # Check that the config object contains the expected keys + assert all(key in config.general for key in expected_general) + + # Check that the default values are as expected + assert all(config.general[key] == expected_general[key] for key in expected_general) + # Check the extractor parameters expected_extractor = { "name": "superpoint", "nms_radius": 3, @@ -75,6 +77,7 @@ def test_valid_basic_arguments(data_dir): for key in expected_extractor ) + # Check the matcher parameters expected_matcher = { "name": "lightglue", "n_layers": 9,