No AC3 playback on Tizen 3.0 (2017) devices

[stefangehrer]

Have you read the FAQ and checked for duplicate open issues?
yes

If the problem is related to FairPlay, have you read the tutorial?

n/a

What version of Shaka Player are you using?

4.12.8

Can you reproduce the issue with our latest release version?
yes

Can you reproduce the issue with the latest code from main?
yes

Are you using the demo app or your own custom app?
custom app

If custom app, can you reproduce the issue using our demo app?
not run on target device, for the demo app in desktop browser we have no AC3 decoding capability

What browser and OS are you using?
Chromium 47 on Tizen 3.0

For embedded devices (smart TVs, etc.), what model and firmware version are you using?
Samsung UE55MU6199 with firmware T-KTMDEUC-1400.3

What are the manifest and license server URIs?

An account for our German TV streaming service waipu.tv was provided to @joeyparrish and can be used to generate streams with 5h validity and no GeoIP restrictions. For other interested parties we could send something up.

What configuration are you using? What is the output of player.getNonDefaultConfiguration()?

see attached

shaka_configuration.txt

What did you do?

Start streaming waipu.tv linear service with dynamic DASH manifests containing AC3 audio track, e.g. the first channel "Das Erste"

What did you expect to happen?
When selecting AC3 in the language/subtitle menu, audio should play back

What actually happened?

When selecting AC3 in the language/subtitle menu, playback continues but audio stays mute

Are you planning to send a PR to fix it?
attached is a patch we used against 4.3.x players: similar to what happens in the manifests with replacing ac-3 with ec-3
e.g. restored here 83c57ee
we do the same manipulation in the init segments, replacing the ISOBMFF atom's four letter codes ac-3 by ec-3 and dac3 by dec3

We wondered if this approach would have any chance of being upstreamed, as it is a rather crude workaround. If so we could try to port it to a recent version and send a PR.

[stefangehrer]

Somehow I couldn't attach the patch, so here it is in comment

index a06a998a8..4dba2abf9 100644
--- a/lib/media/content_workarounds.js
+++ b/lib/media/content_workarounds.js
@@ -287,6 +287,48 @@ shaka.media.ContentWorkarounds = class {
       boxView.setUint32(ContentWorkarounds.BOX_SIZE_OFFSET_, newBoxSize);
     }
   }
+
+  /**
+   * Transform the init segment into a new init segment buffer that indicates
+   *
+   * Should only be called for MP4 init segments, and only on platforms that
+   * need this workaround.
+   *
+   * @param {!BufferSource} initSegmentBuffer
+   * @return {!Uint8Array}
+   */
+  static fakeEAC3(initSegmentBuffer) {
+    const ContentWorkarounds = shaka.media.ContentWorkarounds;
+    const initSegment = shaka.util.BufferUtils.toUint8(initSegmentBuffer);
+    const ancestorBoxes = [];
+
+    const onSimpleAncestorBox = (box) => {
+      ancestorBoxes.push({start: box.start, size: box.size});
+      shaka.util.Mp4Parser.children(box);
+    };
+
+    new shaka.util.Mp4Parser()
+        .box('moov', onSimpleAncestorBox)
+        .box('trak', onSimpleAncestorBox)
+        .box('mdia', onSimpleAncestorBox)
+        .box('minf', onSimpleAncestorBox)
+        .box('stbl', onSimpleAncestorBox)
+        .box('stsd', (box) => {
+          ancestorBoxes.push({start: box.start, size: box.size});
+          const stsdBoxView = shaka.util.BufferUtils.toDataView(
+              initSegment, box.start);
+          for (let i=0; i<box.size; i++) {
+            const codecTag = stsdBoxView.getUint32(i);
+            if (codecTag == ContentWorkarounds.BOX_TYPE_AC_3_) {
+              stsdBoxView.setUint32(i, ContentWorkarounds.BOX_TYPE_EC_3_);
+            } else if (codecTag == ContentWorkarounds.BOX_TYPE_DAC3_) {
+              stsdBoxView.setUint32(i, ContentWorkarounds.BOX_TYPE_DEC3_);
+            }
+          }
+        }).parse(initSegment);
+
+    return initSegment;
+  }
 };
 
 /**
@@ -422,3 +466,8 @@ shaka.media.ContentWorkarounds.BOX_TYPE_ENCV_ = 0x656e6376;
  * @private
  */
 shaka.media.ContentWorkarounds.BOX_TYPE_ENCA_ = 0x656e6361;
+
+shaka.media.ContentWorkarounds.BOX_TYPE_AC_3_ = 0x61632d33;
+shaka.media.ContentWorkarounds.BOX_TYPE_DAC3_ = 0x64616333;
+shaka.media.ContentWorkarounds.BOX_TYPE_EC_3_ = 0x65632d33;
+shaka.media.ContentWorkarounds.BOX_TYPE_DEC3_ = 0x64656333;
diff --git a/lib/media/media_source_engine.js b/lib/media/media_source_engine.js
index 82558e2ac..c3d9671ba 100644
--- a/lib/media/media_source_engine.js
+++ b/lib/media/media_source_engine.js
@@ -1424,6 +1424,14 @@ shaka.media.MediaSourceEngine = class {
       segment = shaka.media.ContentWorkarounds.fakeEncryption(segment);
     }
 
+    if (isInitSegment &&
+        shaka.util.Platform.requiresEAC3InitSegments() &&
+        shaka.util.MimeUtils.getContainerType(
+            this.sourceBufferTypes_[contentType]) == 'mp4') {
+      shaka.log.debug('Forcing fake EAC3 information in init segment.');
+      segment = shaka.media.ContentWorkarounds.fakeEAC3(segment);
+    }
+
     return segment;
   }
 
diff --git a/lib/util/platform.js b/lib/util/platform.js
index adaff9635..68a60428e 100644
--- a/lib/util/platform.js
+++ b/lib/util/platform.js
@@ -410,6 +410,20 @@ shaka.util.Platform = class {
     return Platform.isTizen() || Platform.isXboxOne();
   }
 
+  /**
+   * Returns true if the platform requires AC-3 signalling in init
+   * segments to be replaced with EAC-3 signalling.
+   * For such platforms, MediaSourceEngine will attempt to work
+   * around it by inserting fake EAC-3 signalling into
+   * initialization segments.
+   *
+   * @return {boolean}
+   */
+  static requiresEAC3InitSegments() {
+    const Platform = shaka.util.Platform;
+    return Platform.isTizen();
+  }
+
   /**
    * Returns true if MediaKeys is polyfilled
    *

[avelad]

Let me try some things out in our lab and if it works I'm okay with you submitting a PR with this. Give me a day to test it out. Thanks!

[avelad]

I'm going to try a similar fix for our transmuxer, #7956

[avelad]

@stefangehrer It seems that changing the initialization segment from ac-3 to ec-3 does not solve our problems in the transmuxer. I'm checking and we don't have any assets with AC-3 MP4 to test in Tizen, so I'd like to see your PR that includes a playback test of this specific case (a 4-second sample with two segments is sufficient for the test (the test does not have to have a video))

[avelad]

BTW, if you want to find a way to make our AC-3 transmuxer work with Tizen, you are also welcome (see #7956) :)

[avelad]

@stefangehrer I have tried your change in https://github.com/shaka-project/shaka-player/pull/7964/files and it doesn't work.

[stefangehrer]

Thanks @avelad for your support and test. I can confirm that when I use the provided change on 4.12.x branch playback does not work, while on 4.3.x it does (but we have some other patches in that branch to make the track selection possible at all). I am still investigating if I can find the relevant difference.

[avelad]

All PRs have to be against the main branch, so you would have to test against that branch.

[stefangehrer]

Okay, no problem to go against main branch. I already identified a lot of logs that I did not see in the old, working version. especially

main.js?LnhdmlGPyT0s:1 Using cached results of mediaCapabilities.decodingInfo for key {"audio":{"bitrate":448000,"channels":2,"contentType":"audio/mp4; codecs=\"ec-3\"","samplerate":48000,"spatialRendering":false},"keySystemConfiguration":{"audio":{"robustness":"SW_SECURE_CRYPTO"},"distinctiveIdentifier":"optional","initDataType":"cenc","keySystem":"com.widevine.alpha","persistentState":"optional","sessionTypes":[0:"temporary"],"video":{"robustness":"SW_SECURE_CRYPTO"}},"type":"media-source","video":{"bitrate":7100000,"contentType":"video/mp4; codecs=\"avc1.64002A\"","framerate":50,"height":1080,"width":1920}}

and

main.js?LnhdmlGPyT0s:1 Using full mime type audio/mp4; codecs="ac-3"
main.js?LnhdmlGPyT0s:1 Using full mime type video/mp4; codecs="avc1.64002A"

indicate different behavior.

[avelad]

you change ac-3 to ec-3 too...right?

[stefangehrer]

Yes, that's why I am wondering about the - Using full mime type audio/mp4; codecs="ac-3" - log

[stefangehrer]

If I just hard-replace ac-3 by ec-3 at that place and therefore the addSourceBuffer() gets called with that type I get playback working on my 2017 Tizen TV. If you have a hint where to "properly" do the replacement I would be glad to hear.

addExtraFeaturesToMimeType_(mimeType) {
     const extraFeatures = this.config_.addExtraFeaturesToSourceBuffer(mimeType);
-    const extendedType = mimeType + extraFeatures;
+    let extendedType = mimeType + extraFeatures;
+    extendedType = extendedType.replace('ac-3', 'ec-3');
     shaka.log.debug('Using full mime type', extendedType);
 
     return extendedType;

[avelad]

@stefangehrer See #7964 I'm testing a lot of things...

[avelad]

@Boubalou FYI!

[stefangehrer]

I took the change from StreamUtils.getCorrectAudioCodecs() but it seems that has no effect on what ends up on media's addSourceBuffer() call, will look further.

[avelad]

@stefangehrer With the PR changes the AC-3 transmuxer is working! for raw AC-3 and for AC-3 in TS but not when muxed in TS... :(

So we are moving in the right direction!

[stefangehrer]

I added the StreamUtils.getCorrectAudioCodecs() call also to the initialisation of the source buffer, please have a look
#7969

[avelad]

@stefangehrer LGTM, you would need to sign the CLA and add a unit test at https://github.com/shaka-project/shaka-player/blob/main/test/media/content_workarounds_unit.js