Skip to content

Commit

Permalink
Unify static linking for all operating systems and improve README
Browse files Browse the repository at this point in the history
This commit unifies the code for static linking under all operating
systems, with a focus on fixing problems for Windows:

1.  All functionality for pkg-config is now enabled for Windows as well.
    For example, a vcpkg-built pkgconf works great.  Therefore, it seems
    there is no reason to prevent users from using pkg-config on Windows
    if they want to use it.

2.  When not using pkg-config on Windows, and statically linking against
    FFmpeg libraries, many system libraries will also need to be linked.
    A list of known system libraries will now be linked in such
    situations.  It's best to use pkg-config, but this will at least
    reduce the probability of linker errors.  Note that we also have to
    do this for vcpkg as well.

3.  Code duplication between Windows and other operating systems has
    been eliminated.

4.  The statik flag is now correctly passed to pkg_config when in the
    static_linking function.

5.  The GitHub Actions workflow is updated to test the new possibilities
    on Windows.  The example program is also run, which revealed that
    the existing vcpkg-based way of finding FFmpeg was missing several
    system libraries for linking.

6.  Make significant improvements to the README.  More detailed
    information about environment variables, features, how the build
    script works, vcpkg triples, and more is included to help users
    work with the crate.

Improving the functionality of dynamic linking is an obvious next step,
but is out of scope of this commit.  It's likely that a single, large
function can eventually be used for both static and dynamic linking.
  • Loading branch information
JohnstonJ committed Sep 29, 2024
1 parent c0ee83f commit fd84ee9
Show file tree
Hide file tree
Showing 4 changed files with 278 additions and 146 deletions.
55 changes: 46 additions & 9 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ jobs:

- name: Cache vcpkg
id: cache
uses: actions/cache@v3
uses: actions/cache@v4
with:
path: |
${{ github.workspace }}/vcpkg
Expand Down Expand Up @@ -327,20 +327,40 @@ jobs:
- target: "x86_64-pc-windows-msvc"
vcpkg_triplet: "x64-windows-static"
rustflags: "-Ctarget-feature=+crt-static"
linking: "static"
probe_method: "vcpkg"
- target: "x86_64-pc-windows-msvc"
vcpkg_triplet: "x64-windows-static-md"
linking: "static"
probe_method: "vcpkg"
- target: "x86_64-pc-windows-msvc"
vcpkg_triplet: "x64-windows"
dynamic: true
linking: "dynamic"
probe_method: "vcpkg"

- target: "i686-pc-windows-msvc"
vcpkg_triplet: "x86-windows-static"
rustflags: "-Ctarget-feature=+crt-static"
linking: "static"
probe_method: "vcpkg"
- target: "i686-pc-windows-msvc"
vcpkg_triplet: "x86-windows-static-md"
linking: "static"
probe_method: "vcpkg"
- target: "i686-pc-windows-msvc"
vcpkg_triplet: "x86-windows"
dynamic: true
linking: "dynamic"
probe_method: "vcpkg"

# Additional ways of telling the build script to find FFmpeg:
- target: "x86_64-pc-windows-msvc"
vcpkg_triplet: "x64-windows-static-md"
linking: "static"
probe_method: "dir_vars"
- target: "x86_64-pc-windows-msvc"
vcpkg_triplet: "x64-windows-static-md"
linking: "static"
probe_method: "pkg_config"
fail-fast: false
steps:
- uses: actions/checkout@v4
Expand All @@ -358,7 +378,7 @@ jobs:

- name: Cache vcpkg
id: cache
uses: actions/cache@v3
uses: actions/cache@v4
with:
path: |
${{ github.workspace }}/vcpkg
Expand All @@ -377,19 +397,36 @@ jobs:
- name: Set env
shell: bash
run: |
if [ '${{ matrix.config.dynamic }}' != '' ]; then
if [ '${{ matrix.config.linking }}' == 'dynamic' ]; then
echo "VCPKGRS_DYNAMIC=1" >> $GITHUB_ENV
fi
if [ '${{ matrix.config.probe_method }}' == 'vcpkg' ]; then
echo "VCPKG_ROOT=${{ github.workspace }}/vcpkg" >> $GITHUB_ENV
echo "VCPKG_DEFAULT_TRIPLET=${{ matrix.config.vcpkg_triplet }}" >> $GITHUB_ENV
echo "FEATURES=--features link_vcpkg_ffmpeg" >> $GITHUB_ENV
elif [ '${{ matrix.config.probe_method }}' == 'dir_vars' ]; then
echo "FFMPEG_INCLUDE_DIR=${{ github.workspace }}/vcpkg/installed/"\
"${{ matrix.config.vcpkg_triplet }}/include" >> $GITHUB_ENV
echo "FFMPEG_LIBS_DIR=${{ github.workspace }}/vcpkg/installed/"\
"${{ matrix.config.vcpkg_triplet }}/debug/lib" >> $GITHUB_ENV
echo "FEATURES=" >> $GITHUB_ENV
elif [ '${{ matrix.config.probe_method }}' == 'pkg_config' ]; then
echo "PKG_CONFIG=${{ github.workspace }}/vcpkg/installed/"\
"x64-windows/tools/pkgconf/pkgconf.exe" >> $GITHUB_ENV
echo "FFMPEG_PKG_CONFIG_PATH=${{ github.workspace }}/vcpkg/installed/"\
"${{ matrix.config.vcpkg_triplet }}/debug/lib/pkgconfig" >> $GITHUB_ENV
echo "FEATURES=" >> $GITHUB_ENV
fi
- name: Binding build
- name: Binding build and run example
shell: bash
env:
VCPKG_ROOT: ${{ github.workspace }}/vcpkg
RUSTFLAGS: ${{ matrix.config.rustflags }}
VCPKG_DEFAULT_TRIPLET: ${{ matrix.config.vcpkg_triplet }}
LIBCLANG_PATH: ${{ github.workspace }}/clang/lib
LLVM_CONFIG_PATH: ${{ github.workspace }}/clang/bin/llvm-config
run: cargo build --features link_vcpkg_ffmpeg --target ${{ matrix.config.target }} --verbose
run: |
cargo build $FEATURES --target ${{ matrix.config.target }} --verbose
cargo run $FEATURES --target ${{ matrix.config.target }} --example slice --verbose
build_dynamic_and_test_ubuntu:
runs-on: ubuntu-latest
Expand Down
4 changes: 1 addition & 3 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,8 @@ libc = "0.2"
bindgen = "0.70"
camino = "1.1"
once_cell = "1.12"
vcpkg = { version = "0.2", optional = true }

[target.'cfg(not(windows))'.build-dependencies]
pkg-config = "0.3"
vcpkg = { version = "0.2", optional = true }

[features]
# Probe and link FFmpeg with pkg-config
Expand Down
124 changes: 97 additions & 27 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,54 +10,124 @@ Cross platform FFI bindings for FFmpeg internal libraries. This is a crate that:
1. Links FFmpeg libraries for you.
2. Generates Rust binding for FFmpeg libraries.

## Getting started:
## Getting started

To use this crate, you need to set several environment variables.
This create requires configuration before you can start using it. To quickly get started, follow the steps below.

### The simplest usage:
For more advanced usage, consult the [reference section](#reference) below.

#### Linux, macOS..(*nix)
Before beginning to build FFmpeg, you should install LLVM first. This will enable Rusty FFmpeg to generate Rust bindings from the C header files. The `bindgen` documentation has [good instructions](https://rust-lang.github.io/rust-bindgen/requirements.html) on how to install prebuilt LLVM binaries for your platform. You can also build LLVM from source, such as using vcpkg, but this can take hours on all but the most well-equipped computers.

If you have FFmpeg installed with package manager, import `rusty_ffmpeg` with feature `link_system_ffmpeg`. Then it should work.
### Linux, macOS..(*nix)

If you built FFmpeg from source, set `FFMPEG_PKG_CONFIG_PATH` to the path of the generated FFmpeg `pkg-config` directory. Then it should work.
If you have FFmpeg installed with your system package manager, import `rusty_ffmpeg` with feature `link_system_ffmpeg` and the feature for your installed FFmpeg version. The build script will use pkg-config to probe for the system-installed FFmpeg.

#### Windows
If you'd like to use an FFmpeg that you built from source, set `FFMPEG_PKG_CONFIG_PATH` to the path of the generated `pkgconfig` directory. The build script will use pkg-config to probe in that directory. You may also use `vcpkg`, as described below: the instructions should be similar to Windows.

`rusty_ffmpeg` can link FFmpeg using `vcpkg`:
1. Install [`vcpkg`](https://github.com/microsoft/vcpkg), check [documentation of the vcpkg *crate*](https://docs.rs/vcpkg) for the environment variables to set.
2. Import `rusty_ffmpeg` with feature `link_vcpkg_ffmpeg`, Then it should work.
### Windows

### Fine-grained usage:
One of the easiest ways to get started on Windows is to build FFmpeg using [`vcpkg`](https://github.com/microsoft/vcpkg), which provides a [port for FFmpeg](https://vcpkg.io/en/package/ffmpeg).

You need to set several environment variables for both the linking and binding generating procedures.
#### Classic mode of vcpkg

#### To link prebuilt libraries:
1. Install vcpkg according to the [instructions](https://learn.microsoft.com/en-us/vcpkg/get_started/get-started?pivots=shell-powershell#1---set-up-vcpkg) (the first step of cloning and bootstrapping).
2. Build and install ffmpeg: `vcpkg install ffmpeg --triplet=<value>`, where `triplet` is set to an appropriate value for your project ([see below](#vcpkg-triplets)).
3. Check [documentation of the vcpkg *crate*](https://docs.rs/vcpkg) for the environment variables to set.
4. Import `rusty_ffmpeg` with feature `link_vcpkg_ffmpeg` and the feature corresponding to the FFmpeg version that you built.

1. Dynamic linking with pre-built dylib: Set `FFMPEG_DLL_PATH` to the path of `dll` or `so` files. (Windows: Put corresponding `.lib` file next to the `.dll` file.)
You may want to look into [`cargo-vcpkg`](https://crates.io/crates/cargo-vcpkg), which can automate these steps.

2. Static linking with pre-built staticlib: Set `FFMPEG_LIBS_DIR` to the path of FFmpeg pre-built libs directory.
#### Manifest mode

#### To generate bindings:
When using [manifest mode](https://learn.microsoft.com/en-us/vcpkg/concepts/manifest-mode), you create a `vcpkg.json` file in your repository that declares what vcpkg ports your project requires. vcpkg then builds and installs the packages in a `vcpkg_installed` directory that is local to your repository.

1. Compile-time binding generation([requires the `Clang` dylib](https://github.com/KyleMayes/clang-sys/blob/c9ae24a7a218e73e1eccd320174349eef5a3bd1a/build.rs#L23)): Set `FFMPEG_INCLUDE_DIR` to the path of the header files for binding generation.
1. Create a `vcpkg.json` file for your project by following instructions in vcpkg's documentation. In addition to FFmpeg, make sure you also install the [`pkgconf`](https://vcpkg.io/en/package/pkgconf) port as a host tool.
2. Build and install your manifest: `vcpkg install --triplet=<value>`, where `triplet` is set to an appropriate value for your project ([see below](#vcpkg-triplets)).
3. Set the `PKG_CONFIG` environment variable to the vcpkg-built `pkgconf.exe` tool within the built `tools/` directory. Set the `PKG_CONFIG_PATH` to the `pkgconfig` directory that was installed within the built `lib/` directory.
4. Import `rusty_ffmpeg` with the feature corresponding to the FFmpeg version that you built.

2. Use your prebuilt binding: Set `FFMPEG_BINDING_PATH` to the pre-built binding file. The pre-built binding is usually copied from the `OUT_DIR` of the compile-time binding generation, using it will prevent the need to regenerate the same binding file repeatedly.
## Reference

### Linking FFmpeg installed by package manager on (*nix)
Basic usage of Rusty FFmpeg requires that you:

You can link FFmpeg libraries installed by package manager by enabling feature `link_system_ffmpeg` (which uses pkg-config underneath).
- Set the feature for the FFmpeg version that you are using (see [Features](#features), below).
- [Instruct the build script](#how-the-build-script-finds-ffmpeg) how to find FFmpeg.

### Linking FFmpeg installed by vcpkg
### How the build script finds FFmpeg

You can link FFmpeg libraries installed by vcpkg by enabling feature `link_vcpkg_ffmpeg` on Windows, macOS, and Linux.
There are several ways that FFmpeg can be found and configured using the [environment variables](#environment-variables) and [features](#features). There is no default method for finding FFmpeg: you must choose one or more of these methods to utilize.

### Use a specific FFmpeg version
Unless the `FFMPEG_DLL_PATH` environment variable is set, static linking will be done.

- Do nothing when you are using FFmpeg `4.*`
- Enable `ffmpeg5` feature when you are using FFmpeg `5.*`
- Enable `ffmpeg6` feature when you are using FFmpeg `6.*`
- Enable `ffmpeg7` feature when you are using FFmpeg `7.*`
#### Static linking

When the build script is in static linking mode, it will try to statically link with FFmpeg in the following order:

1. First, if `FFMPEG_PKG_CONFIG_PATH` is set, pkg-config or pkgconf is used to probe for FFmpeg by using the `.pc` files in the given directory. The `PKG_CONFIG` and `FFMPEG_BINDING_PATH` environment variables will also be used if they are set.
2. If `FFMPEG_LIBS_DIR` is set, then FFmpeg libraries from that directory will be used. No probing with pkg-config will take place. You must also set the `FFMPEG_INCLUDE_DIR` or the `FFMPEG_BINDING_PATH` environment variable in this case.
3. If the `link_system_ffmpeg` feature is enabled, then pkg-config will be used to probe for an FFmpeg package installed by your system package manager.
4. If the `link_vcpkg_ffmpeg` feature is enabled, then the [vcpkg](https://docs.rs/vcpkg) crate will be used to probe for FFmpeg.
- Note that as of this writing, the [published crate does not support](https://github.com/mcgoo/vcpkg-rs/issues/41) using vcpkg in [manifest mode](https://learn.microsoft.com/en-us/vcpkg/concepts/manifest-mode), so you may want to consider setting `PKG_CONFIG` and `FFMPEG_PKG_CONFIG_PATH` to the corresponding file/directories installed by vcpkg in that case.

Both `link_system_ffmpeg` and `link_vcpkg_ffmpeg` features can be enabled simultaneously. If system pkg-config probing fails, then it will fallback to vcpkg probing.

#### Dynamic linking

At this time, the build script locates a dynamically-linked FFmpeg using only one way. Probing using pkg-config or vcpkg is not currently supported.

To dynamically link with FFmpeg, the following environment variables must be set:

- `FFMPEG_DLL_PATH`
- `FFMPEG_INCLUDE_DIR` or `FFMPEG_BINDING_PATH`

At this time, more advanced search methods such as those used for static linking are not supported.

### Environment variables

Several aspects of the build are controlled by environment variables:

| Variable | Purpose |
| -------- | ------- |
| `FFMPEG_BINDING_PATH` | Path to a pre-built `binding.rs` file so as to avoid repeatedly regenerating the same binding file. Include files will be ignored in this case; the variable takes precedence over `FFMPEG_INCLUDE_DIR` or any include files discovered using pkg-config. It can be copied from the `OUT_DIR` of a previous build that generated the bindings. |
| `FFMPEG_DLL_PATH` | Path to the FFmpeg dynamically linked library file. The lib file is expected to be alongside it. |
| `FFMPEG_INCLUDE_DIR` | Path to directory containing FFmpeg include files. Bindings will be generated using bindgen. |
| `FFMPEG_LIBS_DIR` | Path to directory containing FFmpeg library files. |
| `FFMPEG_PKG_CONFIG_PATH` | Path to directory containing `.pc` files to probe with pkg-config. |
| `PKG_CONFIG` | Path to pkg-config executable file, if pkg-config is not in your `PATH`. The pkgconf tool can also be used. |

### Features

This crate has several features which can be configured:

| Feature | Purpose |
| ------- | ------- |
| `ffmpeg5` | Compile with support for FFmpeg `5.*` |
| `ffmpeg6` | Compile with support for FFmpeg `6.*` |
| `ffmpeg7` | Compile with support for FFmpeg `7.*` |
| `link_system_ffmpeg` | Enable probing for FFmpeg installed by your system package manager using pkg-config. |
| `link_vcpkg_ffmpeg` | Enable probing for FFmpeg using the [vcpkg](https://docs.rs/vcpkg) crate. |

None of the features are enabled by default. If no `ffmpeg*` feature is chosen, then support for FFmpeg `4.*` will be compiled.

### vcpkg triplets

The vcpkg [triplet](https://learn.microsoft.com/en-us/vcpkg/concepts/triplets) controls the CPU architecture, operating system, runtime library, etc. It is very important that you choose a triplet that is known to be compatible with your Rust toolchain targets and configuration in order to avoid undefined behavior.

This table provides a quick reference for what vcpkg triplet to choose:

| Rust toolchain target | FFmpeg linking | CRT linking | vcpkg triplet |
| --------------------- | -------------- | ----------- | ------------- |
| `x86_64-pc-windows-msvc` | Static | Static | `x64-windows-static` |
| `x86_64-pc-windows-msvc` | Static | Dynamic | `x64-windows-static-md` |
| `x86_64-pc-windows-msvc` | Dynamic | Dynamic | `x64-windows` |
| `x86_64-unknown-linux-gnu` | Static | Dynamic | `x64-linux` |
| `x86_64-unknown-linux-gnu` | Dynamic | Dynamic | `x64-linux-dynamic` |
| `aarch64-apple-darwin` | Static | Dynamic | `arm64-osx` |
| `aarch64-apple-darwin` | Dynamic | Dynamic | `arm64-osx-dynamic` |

- **Rust toolchain target**: View Rust toolchain targets using `rustup target list`.
- **CRT linking**: The C runtime can be dynamically or statically linked to the code compiled by Rust. The [`crt-static`](https://doc.rust-lang.org/reference/linkage.html#static-and-dynamic-c-runtimes) target feature controls this; the default varies by target. On Windows, macOS, and Linux with glibc, dynamic linking is the Rust default.
- **vcpkg triplet**: The vcpkg triplet that you should use for the given Rust target and CRT linking. Typically set using the [`--triplet`](https://learn.microsoft.com/en-us/vcpkg/commands/common-options#triplet) parameter.

## Attention

Expand Down
Loading

0 comments on commit fd84ee9

Please sign in to comment.