Video Sometimes Not Shown when using Apizee #583

AleksandarTokarev · 2020-10-15T09:46:21Z

Hello,
We are seeing issues with the Video on various IPhone devices running on different IOS Versions (12-14 at the time)
When app is initially opened (from killed state) and a video call is established, the IOS video is OK and running.
After repeating the call with the app open several times, video starts to not show.
There are several states that i am seeing:

Video sometimes does not show only on the IOS device (white screen)- but it shows for the other person on the call
Video sometimes does not show for both sides (white screen instead of video)
Video sometimes is black for both sides

In all of the cases, Audio is working fine.

We are using Apizee as WebRTC provider (https://www.apizee.com/)

Also in the logs sometimes we are seeing this error

Error: getStats() must be called with null or a valid MediaStreamTrack instance as argument", url: "http://localhost:12710/plugins/cordova-plugin-iosrtc/dist/cordova-plugin-iosrtc.js", lineNo: 2263, columnNo: 106, error: {}, stack: "http://localhost:12710/plugins/cordova-plugin-iosrtc/dist/cordova-plugin-iosrtc.js:2263:18\nhttps://cloud.apizee.com/apiRTC/apiRTC-latest.min.js:2:651202"

Versions affected

Plugin version (6.0.11, 6.0.14, 8.0.0-RC2):
iOS version (13.X):
Xcode version (11.6):

Steps to reproduce

Start a Video Call several times in a row

Expected results

Video should always show up

Actual results

Video sometimes does not show up

The text was updated successfully, but these errors were encountered:

hthetiot · 2020-10-16T19:26:56Z

Sorry this is most likely related to Apizee usage of WebRTC itself that is not in the officially supported list of libraries.
We are investigating MediaStream recyclying issue that may be related to what you experience but we also found it itmostly depend the way you use WebRTC and manage MediaStream (stopping them) and VideoMediaElement (empty them).

" I have provided steps to reproduce (e.g. sample code or updated extra/renderer-and-libwebrtc-tests.js file)."
https://github.com/cordova-rtc/cordova-plugin-iosrtc/blob/master/.github/ISSUE_TEMPLATE.md

You did not provide step to reproduce your issue using Apizee API.

"If the checkboxes above are not checked (which you do after the issue is posted), the issue will be closed, removing this checkbox will result in automatic closed issue."

You removed checkboxes and therefor the issue is closed, as you would have seen if you had read the ISSUE requirement.

You can always update this issue to match requirement (mainly "I have provided steps to reproduce"), but it unlikely I will help since i not paying or using Apizee and it will be to Apizee itself or the community to assist you with a fix once you updated the issue and its re-openned.

hthetiot · 2020-10-16T19:30:08Z

Error: getStats() must be called with null or a valid MediaStreamTrack instance as argument", url: "http://localhost:12710/plugins/cordova-plugin-iosrtc/dist/cordova-plugin-iosrtc.js", lineNo: 2263, columnNo: 106, error: {}, stack: "http://localhost:12710/plugins/cordova-plugin-iosrtc/dist/cordova-plugin-iosrtc.js:2263:18\nhttps://cloud.apizee.com/apiRTC/apiRTC-latest.min.js:2:651202"

Visibly they dont know how GetStats work since it does take a MediaStream or null... since 2014 (6 years ago):

https://developer.mozilla.org/en-US/docs/Web/API/RTCPeerConnection/getStats

hthetiot · 2020-10-16T20:38:01Z

Never heard about white screen before in my years of webrtc, i guess that Apizee MCU/SFU doing that...

hthetiot · 2020-10-16T20:39:05Z

It will be inestigated once #575 that is most likely a duplicate

AleksandarTokarev · 2020-10-17T09:37:58Z

@hthetiot
Apizee = ApiRTC. This is stated in the "Who Uses It" section
https://github.com/cordova-rtc/cordova-plugin-iosrtc/blob/master/WHO_USES_IT.md
I guess it should be supported by this plugin?

One thing we noticed is, this thing always happens on the Callee side. If we have an user with IPHONE calling an user with IPHONE, the video and the stream ALWAYS works on the Caller side (means caller gets his own video and the Caller video is displayed on the Callee). On the callee side, the first 2-3 times it works but after that the video does not (means the Calle e video is not shown on Callee side and Callee video is NOT shown on Caller side as well).
We are basically using the 1 on 1 call scenario specific here
https://dev.apirtc.com/tutorials/peertopeer/call

 .on("localStreamAvailable", function (stream) {
            console.log('localStreamAvailable');
            //document.getElementById('local-media').remove();
            addStreamInDiv(stream, 'local-container', 'local-media-' + stream.getId(), {width : "160px", height : "120px"}, true);
            stream
                .on("stopped", function () { //When client receives an screenSharing call from another user
                    console.error("Stream stopped");
                    $('#local-media-' + stream.getId()).remove();
                });
        })
        .on("streamAdded", function (stream) {
            console.log('stream :', stream);
            addStreamInDiv(stream, 'remote-container', 'remote-media-' + stream.getId(), {width : "640px", height : "480px"}, false);
        })
        .on('streamRemoved', function (stream) {
            // Remove media element
            document.getElementById('remote-media-' + stream.getId()).remove();
        })
        .on('userMediaError', function (e) {
            console.log('userMediaError detected : ', e);
            console.log('userMediaError detected with error : ', e.error);

            //Checking if tryAudioCallActivated
            if (e.tryAudioCallActivated === false) {
                $('#hangup-' + call.getId()).remove();
            }
        })
        .on('desktopCapture', function (e) {
            console.log('desktopCapture event : ', e);
            $('#hangup-' + call.getId()).remove();
        })
        .on('hangup', function () {
            $('#hangup-' + call.getId()).remove();
        });

Also what you posted above, i can confirm that the MediaStreamObjects are piling up if subsequent calls are being made.
I tried to do cordova.plugins.iosrtc.mediaStreams = undefined but nothing changed, the behavior is same

hthetiot · 2020-10-17T11:24:32Z

@hthetiot
Apizee = ApiRTC. This is stated in the "Who Uses It" section
https://github.com/cordova-rtc/cordova-plugin-iosrtc/blob/master/WHO_USES_IT.md
I guess it should be supported by this plugin

In fact it does not implicitly make them supported, it simply that they use it.

Thank you for providing way to reproduce, let see if 6.0.14 fix your problem, if not we can try to address it in 6.0.15. please try master this way to confirm if incoming 6.0.14 fix some of your problems.

AleksandarTokarev · 2020-10-17T14:26:13Z

I tried the latest master commit. This seems even more unstable for me now. Before the caller side was working OK, now with 6.0.14, that seems unstable as well.

hthetiot · 2020-10-17T15:39:16Z

arf thank you @AleksandarTokarev can you elaborate by what you mean by "even more unstable", this is not clear for me.
master is stable for me.

hthetiot · 2020-10-17T15:40:56Z

Here is 6.0.13 vs master changes for references:
6.0.13.diff.txt

+* Move dist/cordova-plugin-iosrtc.js to www/cordova-plugin-iosrtc.js to match other cordova plugins practice.
+* fix: undefined references #563
+* fix: use video view for correct screencap framing #570
+* fix: cleanup old streams #570
+* implement basic MediaDevices.prototype.getSupportedConstraints SHIM #564
+* fix RTCPeerConnection.prototype.addTrack missing return RTCRtpSender #572

hthetiot · 2020-10-17T15:48:05Z

I think your issue is only when you make multiple calls, and only relate to #575 that will be treated in 6.0.15

AleksandarTokarev · 2020-10-17T16:26:01Z

With the newest master version (which is supposed to be 6.0.14), the behavior is different as you can see in the video below.
Basically, the Remote Stream is not available at all - on both devices. After some calls, on the caller side, the Local Stream is not available as well
https://www.dropbox.com/s/ap7ibahzuacbbke/20201017_182151.mp4?dl=0

And yes, on 6.0.11, this happens when multiple calls are being made.

hthetiot · 2020-10-17T16:43:34Z

Thank you @AleksandarTokarev that more clear to me, I will do what I can to fix this issue on 6.0.15

AleksandarTokarev · 2020-10-17T16:45:07Z

@hthetiot
Here is a video of 6.0.11 behavior. Quite different than 6.0.14.
Here the Remote video is available first 2-3 times, but from 3-4th on, the Callee stream is not available at all
https://www.dropbox.com/s/qr0lx2vfxtevn12/20201017_184053.mp4?dl=0

hthetiot · 2020-10-17T23:09:29Z

@AleksandarTokarev can you try this PR #587
This , I think should change the behavior to a better result. Keep me posted.

AleksandarTokarev · 2020-10-18T08:13:16Z

I tried #587 @hthetiot , it loads the video on both sides all the time now.
Here is a video of how it works now.
https://www.dropbox.com/s/unc8e8v86u7gra5/20201018_100612.mp4?dl=0
One small comment which is not so important - but good if it can be addressed for better UI/UX - would be the black screen not to show initially (if it is possible - check the video in the comment).

Thanks a lot @hthetiot, i really appreciate your help!!!!

hthetiot · 2020-10-18T09:13:31Z

@AleksandarTokarev I think this PR will address the black screen background you see in your video. Will be merged soon.

#586

hthetiot · 2020-10-18T09:14:26Z

Thank you for taking videos that was really useful to understand your issue, also nice UI.

Happy RTC.

AleksandarTokarev · 2020-10-18T09:29:50Z

@hthetiot Is there a way to test #586 and #587 together before being merged into master?
Making a custom branch with both things would be possible, but not sure if the repo is managed like that.
Thanks a lot for all your help!!!!

hthetiot · 2020-10-18T09:31:33Z

I can make a branch for you, give me a moment. Will notify you here.

hthetiot · 2020-10-18T09:43:33Z

@AleksandarTokarev Here is your branch (https://github.com/cordova-rtc/cordova-plugin-iosrtc/tree/AleksandarTokarev) with #586 and #587 together, notice that i have not tested #586 yet, you may try to set the background color of the video tags via CSS in your pages like this video {background-color: purple; } to test it.

cordova plugin remove cordova-plugin-iosrtc --verbose
cordova plugin add https://github.com/cordova-rtc/cordova-plugin-iosrtc#AleksandarTokarev --verbose
cordova platform remove ios --no-save
cordova platform add ios --no-save

How i did it, in case you want to do custom branch in the future:

git clone [email protected]:cordova-rtc/cordova-plugin-iosrtc.git
cd cordova-plugin-iosrtc
git remote add calebboyd https://github.com/calebboyd/cordova-plugin-iosrtc.git
git fetch calebboyd -a -v
git checkout -b AleksandarTokarev
git merge bugs/PeerConnectionTracks
git merge background
git push --set-upstream origin AleksandarTokarev

Thank you, your testing of #586 will save me some time ;)

hthetiot · 2020-10-18T09:44:26Z

Feel free to add your company/app to https://github.com/cordova-rtc/cordova-plugin-iosrtc/blob/master/WHO_USES_IT.md when you want :)

AleksandarTokarev · 2020-10-18T10:21:29Z

@hthetiot I tried the branch you created for #586 and #587 , it seems that the background-color stuff works fine, but the remote video is not available (as it was in the 6.0.14). Here is a video of that
https://www.dropbox.com/s/hzybgso1qmc51nz/20201018_121745.mp4?dl=0

Also as u can see, sometimes a black screen is shown initially.
I only added background-color: purple; in the code

hthetiot · 2020-10-18T10:57:12Z

Thank you @AleksandarTokarev, the version for you will actually be 6.0.15 aka master since I just landed #586 on master and released 6.0.14.

@calebboyd can you check the branch #586 and rebase/merge on master, I suspect the issue is your "superView" extras view that break HTML over video.

hthetiot · 2020-10-18T11:10:24Z

If you dont mind @AleksandarTokarev try master one most time to make sure we get same result as when you tested #587
Thank you, your testing help a lot.

cordova plugin remove cordova-plugin-iosrtc --verbose
cordova plugin add https://github.com/cordova-rtc/cordova-plugin-iosrtc#master --verbose
cordova platform remove ios --no-save
cordova platform add ios --no-save

AleksandarTokarev · 2020-10-18T11:10:26Z

@hthetiot i tested the latest master (without the last commit u had made just now) and apart from the black screen initially (which i reported in #583 (comment)) , all looks good. Here is the commit that i tested with

hthetiot · 2020-10-18T11:11:33Z

Perfect, yes those commit are for README and sample and does not have impact for you.
I just fixed the jslint issue on render-and-libwebrtc-test.js ;)

AleksandarTokarev · 2020-10-18T11:13:13Z

Thanks a lot for everything. If it is possible to fix the black screen in the future it would be great. Thanks a lot and let me know if i need to do some testing in the future :)

AleksandarTokarev · 2020-10-19T10:53:07Z

@hthetiot We did some more thorough testing with our colleagues today and it seems that the problem still appears (But not as often). Before the changes - after like 3rd time, the Callee video was not available in any sub sequent calls. Now this is not the case, which basically means maybe after 5-6 times, 7th maybe it wont appear (the Callee video), but after that it starts appearing again.

hthetiot · 2020-10-19T15:01:23Z

Unfortunaly i do not reproduce using the iosrtc sample, what would really help to confirm this is an iosrtc issue would to create an index-apizee.js file in the sample to test the iosrtc sample with apizee.

See existing samples:

https://github.com/cordova-rtc/cordova-plugin-iosrtc-sample/tree/master/www/js

Using JsSip and Local and EasyRTC i can make 20 calls without issues.

hthetiot · 2020-10-19T15:01:54Z

Regarding yout black screen; you can also handle it by displaying a loader until you receive video event "canplay".

hthetiot closed this as completed Oct 16, 2020

hthetiot added the boulet label Oct 16, 2020

hthetiot added bug and removed boulet labels Oct 16, 2020

hthetiot added this to the 6.0.15 milestone Oct 16, 2020

hthetiot added the need reporter action label Oct 16, 2020

hthetiot reopened this Oct 17, 2020

hthetiot removed the need reporter action label Oct 17, 2020

hthetiot changed the title ~~Video Sometimes Not Shown~~ Video Sometimes Not Shown when using Apizee Oct 17, 2020

hthetiot added the third party library support label Oct 17, 2020

hthetiot mentioned this issue Oct 17, 2020

Black screen when a second iOS user connects to a Twilio room #576

Closed

3 tasks

hthetiot mentioned this issue Oct 17, 2020

refactor pluginMediaTracks lifecyle in PluginRTCPeerConnection, add canvas drawImage SHIM, add MediaStream(Track)?.clone #587

Merged

hthetiot self-assigned this Oct 17, 2020

hthetiot added the need reporter action label Oct 18, 2020

hthetiot removed the need reporter action label Oct 18, 2020

hthetiot closed this as completed Oct 18, 2020

hthetiot mentioned this issue Oct 18, 2020

feat: use background-color from video element css #586

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Video Sometimes Not Shown when using Apizee #583

Video Sometimes Not Shown when using Apizee #583

AleksandarTokarev commented Oct 15, 2020

hthetiot commented Oct 16, 2020 •

edited

Loading

hthetiot commented Oct 16, 2020

hthetiot commented Oct 16, 2020

hthetiot commented Oct 16, 2020

AleksandarTokarev commented Oct 17, 2020 •

edited

Loading

hthetiot commented Oct 17, 2020 •

edited

Loading

AleksandarTokarev commented Oct 17, 2020

hthetiot commented Oct 17, 2020

hthetiot commented Oct 17, 2020 •

edited

Loading

hthetiot commented Oct 17, 2020

AleksandarTokarev commented Oct 17, 2020 •

edited

Loading

hthetiot commented Oct 17, 2020

AleksandarTokarev commented Oct 17, 2020

hthetiot commented Oct 17, 2020

AleksandarTokarev commented Oct 18, 2020

hthetiot commented Oct 18, 2020

hthetiot commented Oct 18, 2020

AleksandarTokarev commented Oct 18, 2020 •

edited

Loading

hthetiot commented Oct 18, 2020

hthetiot commented Oct 18, 2020

hthetiot commented Oct 18, 2020

AleksandarTokarev commented Oct 18, 2020 •

edited

Loading

hthetiot commented Oct 18, 2020 •

edited

Loading

hthetiot commented Oct 18, 2020

AleksandarTokarev commented Oct 18, 2020 •

edited

Loading

hthetiot commented Oct 18, 2020

AleksandarTokarev commented Oct 18, 2020

AleksandarTokarev commented Oct 19, 2020

hthetiot commented Oct 19, 2020

hthetiot commented Oct 19, 2020

Video Sometimes Not Shown when using Apizee #583

Video Sometimes Not Shown when using Apizee #583

Comments

AleksandarTokarev commented Oct 15, 2020

Versions affected

Steps to reproduce

Expected results

Actual results

hthetiot commented Oct 16, 2020 • edited Loading

hthetiot commented Oct 16, 2020

hthetiot commented Oct 16, 2020

hthetiot commented Oct 16, 2020

AleksandarTokarev commented Oct 17, 2020 • edited Loading

hthetiot commented Oct 17, 2020 • edited Loading

AleksandarTokarev commented Oct 17, 2020

hthetiot commented Oct 17, 2020

hthetiot commented Oct 17, 2020 • edited Loading

hthetiot commented Oct 17, 2020

AleksandarTokarev commented Oct 17, 2020 • edited Loading

hthetiot commented Oct 17, 2020

AleksandarTokarev commented Oct 17, 2020

hthetiot commented Oct 17, 2020

AleksandarTokarev commented Oct 18, 2020

hthetiot commented Oct 18, 2020

hthetiot commented Oct 18, 2020

AleksandarTokarev commented Oct 18, 2020 • edited Loading

hthetiot commented Oct 18, 2020

hthetiot commented Oct 18, 2020

hthetiot commented Oct 18, 2020

AleksandarTokarev commented Oct 18, 2020 • edited Loading

hthetiot commented Oct 18, 2020 • edited Loading

hthetiot commented Oct 18, 2020

AleksandarTokarev commented Oct 18, 2020 • edited Loading

hthetiot commented Oct 18, 2020

AleksandarTokarev commented Oct 18, 2020

AleksandarTokarev commented Oct 19, 2020

hthetiot commented Oct 19, 2020

hthetiot commented Oct 19, 2020

hthetiot commented Oct 16, 2020 •

edited

Loading

AleksandarTokarev commented Oct 17, 2020 •

edited

Loading

hthetiot commented Oct 17, 2020 •

edited

Loading

hthetiot commented Oct 17, 2020 •

edited

Loading

AleksandarTokarev commented Oct 17, 2020 •

edited

Loading

AleksandarTokarev commented Oct 18, 2020 •

edited

Loading

AleksandarTokarev commented Oct 18, 2020 •

edited

Loading

hthetiot commented Oct 18, 2020 •

edited

Loading

AleksandarTokarev commented Oct 18, 2020 •

edited

Loading