token propagation issue

[honkphluxx]

Hi guys!

I am trying to get a local/mapped nginx-vod-module setup to run, pretty much according to your sample given here.

(cleaned) config looks like this

secure_token_akamai $token {
    key 1234;
    acl "$secure_token_baseuri*";
    param_name __hdnea__;
}


server {
    listen 6503;
    listen 6502 ssl;

    server_name  my.domain.com;

    ssl_certificate      ...
    ssl_certificate_key  ...

    vod_segments_base_url http://my.domain.com:1234;
    vod_base_url https://my.domain.com:1235;

    location /hls/ {
        vod_mode mapped;
        alias /var/hls/info/;
        vod hls;

        secure_token $token;
        secure_token_avoid_cookies on;

        secure_token_expires_time 100d;
        secure_token_query_token_expires_time 1h;
        secure_token_types application/vnd.apple.mpegurl;

        secure_token_uri_filename_prefix index;
        secure_token_uri_filename_prefix master;
        secure_token_uri_filename_prefix encryption;

        secure_token_tokenize_segments off;

        akamai_token_validate $arg___hdnea__;
        akamai_token_validate_key 1234;

        akamai_token_validate_uri_filename_prefix encryption;
        akamai_token_validate_uri_filename_prefix index;
        akamai_token_validate_uri_filename_prefix master;


        vod_secret_key "abcd";
        vod_hls_encryption_method aes-128;

        vod_align_segments_to_key_frames on;
        vod_segment_duration 2000;
        vod_live_window_duration 6000;
        vod_max_mapping_response_size 4096;
    }
}

I am trying to serve a stream with adaptive bitrates and multiple languages. A simplified version of my json looks like this

{	"playlistType": "live",
	"segmentBaseTime": 1603022304000,
	"firstClipTime": 1603022304000,
	"expirationTime": 1603042304000,
	"discontinuity": false,
	"durations": [6960000],
	"sequences": [
			{ "clips": [ { "type": "source",
						   "path": "/var/media/1.mp4",
						   "tracks": "v1-a1"
						  } ],
					"language": "ger" },
					
			{ "clips": [ { "type": "source",
						   "path": "/var/media/2.mp4",
						   "tracks": "v1-a2"
						 } ],
					"language": "eng" },

			{ "clips": [ { "type": "source",
						   "path": "/var/media/1.mp4",
						   "tracks": "v1-a1" } ],
					"language": "ger" },
					
			{ "clips": [ { "type": "source",
						   "path": "/var/media/2.mp4",
						   "tracks": "v1-a2" } ],
					"language": "eng" } ] }

Basically, I have one file per bitrate, each with multiple audio tracks. (Talking of it, maybe this confuses the server somehow?)

Requesting the master.m3u8, I retrieve a list of index m3u8s - I dont know all the details of HLS, but I can tell it contains first EXT-X-MEDIA entries pointing to playlists (matching the number of languages I have), and then #EXT-X-STREAM-INF entries - also pointing to playlists (matching the total number of streams).

My last remaining problem seems to be that #EXT-X-MEDIA playlist entries in the master file receive a token, but NOT the #EXT-X-STREAM-INF entries. This results in vlc not working and some Windows Store HLS player just playing audio.

I do not use any names based on vod_multi_uri_suffix - I tried that and it resulted in 404. My current theory is that its rather meant for mpag-dash.

Any help appreciated! If there are any syntax errors in my config files, thats because of my simplifications...

[honkphluxx]

small sidetrack: Is it correct that

akamai_token_validate $arg___hdnea__;

has first 3 underscores, and then 2 underscores, while all other ocurrences of that ID have 2+2 underscores? seems to work like that, and it matches all the examples, but it took some time for me to figure out, and I still dont know how it works.

[honkphluxx]

Doing more investigations, a simpler json that does not create #EXT-X-MEDIA does not help.
#EXT-X-STREAM-INF urls still lack the token.

[honkphluxx]

(TLDR) (having played with my config over and over again, I start to believe its the code..)
For reference, my master .m3u8 looks like this:

EXTM3U

#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio0",NAME="Deutsch",LANGUAGE="de",AUTOSELECT=YES,DEFAULT=YES,CHANNELS="2",URI="https://my.domain.com:123/hls/title/index-f1-a1.m3u8?__hdnea__=st=1603146939~exp=1603233339~acl=/hls/title/*~hmac=2341dc84a6f819a1de0657e3d21e15800909901c9172addacbdaf62dbdb3346c"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio0",NAME="English",LANGUAGE="en",AUTOSELECT=NO,DEFAULT=NO,CHANNELS="2",URI="https://my.domain.com:123/hls/title/index-f2-a2.m3u8?__hdnea__=st=1603146939~exp=1603233339~acl=/hls/title/*~hmac=2341dc84a6f819a1de0657e3d21e15800909901c9172addacbdaf62dbdb3346c"

#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=867868,RESOLUTION=640x346,FRAME-RATE=23.974,CODECS="avc1.4d401e,mp4a.40.2",AUDIO="audio0"
https://my.domain.com:123/hls/title/index-f1-v1-a1.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=867868,RESOLUTION=640x346,FRAME-RATE=23.974,CODECS="avc1.4d401e,mp4a.40.2",AUDIO="audio0"
https://my.domain.com:123/hls/title/index-f2-v1-a2.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1405733,RESOLUTION=842x456,FRAME-RATE=23.974,CODECS="avc1.4d401e,mp4a.40.2",AUDIO="audio0"
https://my.domain.com:123/hls/title/index-f3-v1-a1.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1405733,RESOLUTION=842x456,FRAME-RATE=23.974,CODECS="avc1.4d401e,mp4a.40.2",AUDIO="audio0"
https://my.domain.com:123/hls/title/index-f4-v1-a2.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=2778510,RESOLUTION=1280x692,FRAME-RATE=23.974,CODECS="avc1.4d401f,mp4a.40.2",AUDIO="audio0"
https://my.domain.com:123/hls/title/index-f5-v1-a1.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=2778510,RESOLUTION=1280x692,FRAME-RATE=23.974,CODECS="avc1.4d401f,mp4a.40.2",AUDIO="audio0"
https://my.domain.com:123/hls/title/index-f6-v1-a2.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=5195623,RESOLUTION=1920x1040,FRAME-RATE=23.974,CODECS="avc1.4d4032,mp4a.40.2",AUDIO="audio0"
https://my.domain.com:123/hls/title/index-f7-v1-a1.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=5195623,RESOLUTION=1920x1040,FRAME-RATE=23.974,CODECS="avc1.4d4032,mp4a.40.2",AUDIO="audio0"
https://my.domain.com:123/hls/title/index-f8-v1-a2.m3u8

So why do I get no token in the lower URLS?

2 theories:

• you are using nginx header filters and they stop filtering at the newline (strange that they cross the first newline though)
• EXT-X-STREAM-INF is missing here and/or matching there will not work because the following URL is separated by a newline

[honkphluxx]

My problem sits here:

(while parsing the beginning of a new line)

if (conf->tokenize_segments || conf->encrypt_uri)
				{
					ngx_http_secure_token_url_state_machine_init(
						&ctx->base,
						conf->tokenize_segments,
						STATE_WAIT_NEWLINE,
						0);

					cur_pos--;		// push the current char to the url state machine
				}
				else
				{
					ctx->base.state = STATE_WAIT_NEWLINE;
				}

I am new to this code, but it looks lines not starting with # are skipped with secure_token_encrypt_uri and secure_token_tokenize_segments disabled. I can indeed get those lines tokenized by enabling secure_token_tokenize_segments, which I dont want.

[honkphluxx]

Note to myself (or whoever): A workable change would be this paths of states from a newline:

• line starts with #EXT-X-STREAM-INF -> set an extra bool which is also ORed in line 75.
• line starts with # but its any other tag -> reset the bool again

[honkphluxx]

Not beautiful but seems to do the job for me:

diff --git a/ngx_http_secure_token_m3u8.c b/ngx_http_secure_token_m3u8.c
index 0b412f6..21989b8 100644
--- a/ngx_http_secure_token_m3u8.c
+++ b/ngx_http_secure_token_m3u8.c
@@ -58,6 +58,8 @@ ngx_http_secure_token_m3u8_processor(
        u_char* cur_pos;
        u_char ch;

+       int stream_inf = 0;
+
        for (cur_pos = *pos; cur_pos < buffer_end; cur_pos++)
        {
                ch = *cur_pos;
@@ -72,15 +74,16 @@ ngx_http_secure_token_m3u8_processor(
                        }
                        else if (!isspace(ch))
                        {
-                               if (conf->tokenize_segments || conf->encrypt_uri)
+                               if (conf->tokenize_segments || conf->encrypt_uri || stream_inf)
                                {
                                        ngx_http_secure_token_url_state_machine_init(
                                                &ctx->base,
-                                               conf->tokenize_segments,
+                                               conf->tokenize_segments || stream_inf,
                                                STATE_WAIT_NEWLINE,
                                                0);

                                        cur_pos--;              // push the current char to the url state machine
+                                       stream_inf = 0;
                                }
                                else
                                {
@@ -92,6 +95,10 @@ ngx_http_secure_token_m3u8_processor(
                case STATE_TAG_NAME:
                        if (ch == ':')
                        {
+                               if (ctx->tag_name_len == strlen("EXT-X-STREAM-INF") && strncmp( (char*) ctx->tag_name, "EXT-X-STREAM-INF", strlen("EXT-X-STREAM-INF")) == 0)
+                               {
+                                       stream_inf = 1;
+                               }
                                ctx->base.state = STATE_ATTR_NAME;
                                ctx->attr_name_len = 0;
                        }