Issue when listing (async) FTP folder

[Rayzbam]

I'm working under a .NET 8 Windows environment.

I have a lot of issues to build a generic FTP platform for users and some issues had already been pointed on other topics here, so i'm not going to focus on these ones (still hoping it'll be fixed soon).

Business context : Users are allowed to configure some "watchers" on FTP to get files and move it anywhere they want.
Technical context : Each one of this "watcher" is kind of a singleton in my app, and work with a AsyncFtpClient created one time at the start, and trying to keep it (Noop + Keep-Alive) :

(Let me know if something seems wrong in this connect method in the first place)
As you can see, the first time i'm trying to create the AsyncFtpClient, and the following ones (should) only try to call "AutoConnect" if the "IsConnected" has been setted to false.

But i'm having a new issue, and this one is very annoying because i can't do anything to handle it or try a workaround.

I'm listing working folder simply calling

And having this error which seems internal at FluentFTP

I'm targeting a folder FTP which contains images (a lot of ones, like 47244 at the time i'm writing) so, if there is an error, i would expect a timeout or a closed socket.

This "Object reference not set to an instance of an object." seems a bit different and i would like some help to understand this behavior.

FYI : This process is working 90% of the time.

[FanDjango]

I have a lot of issues to build a generic FTP platform for users and some issues had already been pointed on other topics here, so i'm not going to focus on these ones (still hoping it'll be fixed soon).

Name those issues, please

[Rayzbam]

Hi @FanDjango,

The main point here was not to target FluentFTP (tbh it's the opposite, this helps me a lot to achieve this platform), just wanted to let you know my context and my issues working with multiple and differents FTP.

But if you really are interested, the most common issue (in my case) is mainly due to the AutoConnect :

I could understand if this one was on uploading a file or doing an operation, but when i'm trying to connect (i'm retrying at least one time) feels strange to me.

I have not created an issue for this one because i'm still investigating and found a similar issue which whill give me, maybe, some clues on it => 825

[FanDjango]

For a start, you won't get any help with these screenshots. I would ask you to provide verbose FluentFTP logs.

[FanDjango]

So let's talk about problem #1 (Object reference not set to an instance of an object)

You know, in a log I could see better where and how it happens. And the effing screenshot shows no linenumbers.

Looking at the stack trace, perhaps OpenDataStream opening (connecting) a data stream failed and it tried to use the failed stream (null), which would be a bug in our code. Perhaps....

And then let's talk about problem #2 (Timed out....)

My comment: #825 is ancient and a perfect example of a disorganized and chaotic problem description that wasted everyones time. If you are looking at #825 for clues, you are up the proverbial creek without a paddle.

Oh, and if you feel that perhaps you might need a longer timeout to make connect work more often, note the following two facts:

1. Setting timeouts on windows IS NOT SO EASY. Not our fault. Read this: https://github.com/robinrodricks/FluentFTP/wiki/FTP-Connection#faq_timeoutwindows
2. Async Client Timeout was not being honored (FluentFTP bug) but this has been fixed long ago (early this year?) you would need to search closed issues. You never mentioned the FluentFTP version you are using, are you running something ancient?

So perhaps the two problems are connected

If TCP connects have a history of "flaky" performance on your setup, caused by SLOW SLOW connect which sometimes hits the timeout (read the windows timeout thing again, see above) IT IS NOT ONLY the connect to the server that can fail, it is also whenever a data connection needs to be made, it could timeout too. And if we don't handle that correctly, perhaps, perhaps, that could be an explanation.

I would like to trap that problem, but first you might like to set a longer timeout and see if the problem goes away. That won't help me fix the problem #1, but it will prove the theory. And it would be a solution to your actual problem (works only 80% of the time).

Think about it

[FanDjango]

As you can see, the first time i'm trying to create the AsyncFtpClient, and the following ones (should) only try to call "AutoConnect" if the "IsConnected" has been setted to false.

By the way, "re-using" a FtpClient is allowed. So repeated connect-disconnect sequence should work (they do in my code, for example). Note that you are querying "IsConnected". Please consider using "IsStillConnected()", it will be more accurate by far.

[Rayzbam]

So let's talk about problem #1 (Object reference not set to an instance of an object)

You know, in a log I could see better where and how it happens. And the effing screenshot shows no linenumbers.

Looking at the stack trace, perhaps OpenDataStream opening (connecting) a data stream failed and it tried to use the failed stream (null), which would be a bug in our code. Perhaps....

And then let's talk about problem #2 (Timed out....)

My comment: #825 is ancient and a perfect example of a disorganized and chaotic problem description that wasted everyones time. If you are looking at #825 for clues, you are up the proverbial creek without a paddle.

Oh, and if you feel that perhaps you might need a longer timeout to make connect work more often, note the following two facts:

1. Setting timeouts on windows IS NOT SO EASY. Not our fault. Read this: https://github.com/robinrodricks/FluentFTP/wiki/FTP-Connection#faq_timeoutwindows
2. Async Client Timeout was not being honored (FluentFTP bug) but this has been fixed long ago (early this year?) you would need to search closed issues. You never mentioned the FluentFTP version you are using, are you running something ancient?

So perhaps the two problems are connected

If TCP connects have a history of "flaky" performance on your setup, caused by SLOW SLOW connect which sometimes hits the timeout (read the windows timeout thing again, see above) IT IS NOT ONLY the connect to the server that can fail, it is also whenever a data connection needs to be made, it could timeout too. And if we don't handle that correctly, perhaps, perhaps, that could be an explanation.

I would like to trap that problem, but first you might like to set a longer timeout and see if the problem goes away. That won't help me fix the problem #1, but it will prove the theory. And it would be a solution to your actual problem (works only 80% of the time).

Think about it

1. I can't activate logs on our environment because there are too much logs (and too many FTP at the same time) and i can't reproduce it easily.
   Let me try to reproduce it first with logs activated in debug.

2. For this issue, i think i understood a bit more the use case.
   When connecting to this FTP (this error seems specific to one of our client), the server seems to hang and not send any response.
   Trying on FileZilla and had the connection stuck on :

For a long time.

TL;DR : Might be a bad configuration of the client on our side (issue between FTP/FTPS maybe)

[FanDjango]

For a long time.

Using a telnet client (but make it go to port 21 instead of default 23) or using the native window command shell windows ftp command, try connecting to that server (i. e. use a "stupid") client. Maybe the server is such a strange one, it does not send a welcome message at all any the smart clients like FileZilla or FluentFTP are WAITING for ever.

If you connect with a stupid client, you can try a command like "HELP" immediately, even if there is no welcome message.

On the other hand, if this server sometimes answers, and sometimes does not, then I would assume that is does have some internal problems of its own. Because "connection established" does mean that the IP/TCP SYN-ACK-SYN sequence at least was successful.

[FanDjango]

Could also be, the server is immediately expecting TLS negotiation, try FTPS implicit. Normally it should terminate the connection if it is unhappy, but some servers just hang there and expect YOU to close the connection.

[FanDjango]

But I need to say again:

On the other hand, if this server sometimes answers, and sometimes does not, then I would assume that is does have some internal problems of its own. Because "connection established" does mean that the IP/TCP SYN-ACK-SYN sequence at least was successful.

[Rayzbam]

I'm trying to have a generic code to handle multiple use cases (i know it seems impossible, but i'm still trying).
Is sending a "HELP" command can be interpreted by all servers easily ?
How can i send this HELP command when "AutoConnect" is not responding me and throw an exception ?

When you're saying "try FTPS implicit", you mean EncryptionMode ? If so, i tried every mode tbh, can't find a good fit.

I just succeed to connect on FileZilla but can't find the matching settings to have with FluentFTP :

It seems that plain FTP (not over TLS) is the key here, for FileZilla i mean.

I'm using EncryptionMode.None, is it not the same as plain FTP ? (Nor Explicit Or Implicit)

PS : Sorry we're focusing on the second issue, which is not the one targeted by this topic. Let me know if you need another issue.

[FanDjango]

I'm trying to have a generic code to handle multiple use cases (i know it seems impossible, but i'm still trying).
Is sending a "HELP" command can be interpreted by all servers easily ?
How can i send this HELP command when "AutoConnect" is not responding me and throw an exception ?

No, no, no. I was just trying to give you some ideas for simple manual tests to find out the problem may be.

[FanDjango]

I can't activate logs on our environment...

That makes things difficult

[FanDjango]

I'm using EncryptionMode.None, is it not the same as plain FTP ? (Nor Explicit Or Implicit)

Correct. It is the same.

image
For a long time.

Timeout. Read and understand the above posts and links about "timeout" again. Both these problems could be related to long response times from the server.

And under windows you cannot easily make the timout LONGER (even if you set it longer, it won't work) unless you do special actions in windows.

[Rayzbam]

Correct. It is the same.

Plain FTP connection works 100% on FileZilla, and can't make it connect on my side with EncryptionMode.None
I guess there is something wrong with my AsyncFtpClient.Config but can't find out.

The timeout read socket only appears if i'm trying any kind of connection over TLS on this server, so i don't think it's a real timeout, the server will just never finish the connection sequence.

To be sure to be in plain FTP, should I set some other configs like "SSLProtocols" to None for example ?

[FanDjango]

I guess there is something wrong with my AsyncFtpClient.Config but can't find out.

You have never posted your config here.

[Rayzbam]

I mean the code used in the method posted in the issue.

This one

[FanDjango]

Tired of screenshots instead of pasting code

if (!easyToRead) {
  letSomeOneElseTryToHelp();
}

And with the amount of information available no one can help you more than you could do yourself.
I understand that you don't have logs. Can you not access that server on a special client, where you can log it?
At least consider posting the FileZilla DEBUG log, then, with timestamps included, because you said you could access in FileZilla.
Maybe somebody will chime in and try to help you.