new (#15)

* except and propagate TypeNotFoundError during update handling

* Better document breaking ToS will lead to bans

Closes LonamiWebs#4102.

* Fix KeyError when ID is in cache but queried without mark

Closes LonamiWebs#4084.

* Fix asyncio.CancelledError was being swallowed by inner except

Closes LonamiWebs#4104.

* Add check for asyncio event loop to remain the same

* Update FAQ with more common questions

---------

Co-authored-by: Lonami Exo <[email protected]>

README.rst

@@ -12,6 +12,8 @@ as a user or through a bot account (bot API alternative).
 
     If you have code using Telethon before its 1.0 version, you must
     read `Compatibility and Convenience`_ to learn how to migrate.
+    As with any third-party library for Telegram, be careful not to
+    break `Telegram's ToS`_ or `Telegram can ban the account`_.
 
 What is this?
 -------------
@@ -76,6 +78,8 @@ useful information.
 .. _MTProto: https://core.telegram.org/mtproto
 .. _Telegram: https://telegram.org
 .. _Compatibility and Convenience: https://docs.telethon.dev/en/stable/misc/compatibility-and-convenience.html
+.. _Telegram's ToS: https://core.telegram.org/api/terms
+.. _Telegram can ban the account: https://docs.telethon.dev/en/stable/quick-references/faq.html#my-account-was-deleted-limited-when-using-the-library
 .. _Read The Docs: https://docs.telethon.dev
 
 .. |logo| image:: logo.svg

readthedocs/quick-references/faq.rst

@@ -60,6 +60,16 @@ And except them as such:
 My account was deleted/limited when using the library
 =====================================================
 
+First and foremost, **this is not a problem exclusive to Telethon.
+Any third-party library is prone to cause the accounts to appear banned.**
+Even official applications can make Telegram ban an account under certain
+circumstances. Third-party libraries such as Telethon are a lot easier to
+use, and as such, they are misused to spam, which causes Telegram to learn
+certain patterns and ban suspicious activity.
+
+There is no point in Telethon trying to circumvent this. Even if it succeeded,
+spammers would then abuse the library again, and the cycle would repeat.
+
 The library will only do things that you tell it to do. If you use
 the library with bad intentions, Telegram will hopefully ban you.
 
@@ -75,6 +85,16 @@ would fail. To solve these connection problems, you should use a proxy.
 Telegram may also ban virtual (VoIP) phone numbers,
 as again, they're likely to be used for spam.
 
+More recently (year 2023 onwards), Telegram has started putting a lot more
+measures to prevent spam (with even additions such as anonymous participants
+in groups or the inability to fetch group members at all). This means some
+of the anti-spam measures have gotten more aggressive.
+
+The recommendation has usually been to use the library only on well-established
+accounts (and not an account you just created), and to not perform actions that
+could be seen as abuse. Telegram decides what those actions are, and they're
+free to change how they operate at any time.
+
 If you want to check if your account has been limited,
 simply send a private message to `@SpamBot`_ through Telegram itself.
 You should notice this by getting errors like ``PeerFloodError``,
@@ -178,6 +198,23 @@ won't do unnecessary work unless you need to:
         sender = await event.get_sender()
 
 
+File download is slow or sending files takes too long
+=====================================================
+
+The communication with Telegram is encrypted. Encryption requires a lot of
+math, and doing it in pure Python is very slow. ``cryptg`` is a library which
+containns the encryption functions used by Telethon. If it is installed (via
+``pip install cryptg``), it will automatically be used and should provide
+a considerable speed boost. You can know whether it's used by configuring
+``logging`` (at ``INFO`` level or lower) *before* importing ``telethon``.
+
+Note that the library does *not* download or upload files in parallel, which
+can also help with the speed of downloading or uploading a single file. There
+are snippets online implementing that. The reason why this is not built-in
+is because the limiting factor in the long run are ``FloodWaitError``, and
+using parallel download or uploads only makes them occur sooner.
+
+
 What does "Server sent a very new message with ID" mean?
 ========================================================
 
@@ -241,6 +278,57 @@ same session anywhere else. If you need to use the same account from
 multiple places, login and use a different session for each place you need.
 
 
+What does "Task was destroyed but it is pending" mean?
+======================================================
+
+Your script likely finished abruptly, the ``asyncio`` event loop got
+destroyed, and the library did not get a chance to properly close the
+connection and close the session.
+
+Make sure you're either using the context manager for the client or always
+call ``await client.disconnect()`` (by e.g. using a ``try/finally``).
+
+
+What does "The asyncio event loop must not change after connection" mean?
+=========================================================================
+
+Telethon uses ``asyncio``, and makes use of things like tasks and queues
+internally to manage the connection to the server and match responses to the
+requests you make. Most of them are initialized after the client is connected.
+
+For example, if the library expects a result to a request made in loop A, but
+you attempt to get that result in loop B, you will very likely find a deadlock.
+To avoid a deadlock, the library checks to make sure the loop in use is the
+same as the one used to initialize everything, and if not, it throws an error.
+
+The most common cause is ``asyncio.run``, since it creates a new event loop.
+If you ``asyncio.run`` a function to create the client and set it up, and then
+you ``asyncio.run`` another function to do work, things won't work, so the
+library throws an error early to let you know something is wrong.
+
+Instead, it's often a good idea to have a single ``async def main`` and simply
+``asyncio.run()`` it and do all the work there. From it, you're also able to
+call other ``async def`` without having to touch ``asyncio.run`` again:
+
+.. code-block:: python
+
+    # It's fine to create the client outside as long as you don't connect
+    client = TelegramClient(...)
+
+    async def main():
+        # Now the client will connect, so the loop must not change from now on.
+        # But as long as you do all the work inside main, including calling
+        # other async functions, things will work.
+        async with client:
+            ....
+
+    if __name__ == '__main__':
+        asyncio.run(main())
+
+Be sure to read the ``asyncio`` documentation if you want a better
+understanding of event loop, tasks, and what functions you can use.
+
+
 What does "bases ChatGetter" mean?
 ==================================
 

telethon/client/auth.py

@@ -238,13 +238,14 @@ async def _start(
                 me = await self.sign_in(phone=phone, password=password)
 
         # We won't reach here if any step failed (exit by exception)
-        signed, name = 'Signed in successfully as', utils.get_display_name(me)
+        signed, name = 'Signed in successfully as ', utils.get_display_name(me)
+        tos = '; remember to not break the ToS or you will risk an account ban!'
         try:
-            print(signed, name)
+            print(signed, name, tos, sep='')
         except UnicodeEncodeError:
             # Some terminals don't support certain characters
             print(signed, name.encode('utf-8', errors='ignore')
-                              .decode('ascii', errors='ignore'))
+                              .decode('ascii', errors='ignore'), tos, sep='')
 
         return self
 

telethon/client/telegrambaseclient.py

@@ -399,6 +399,7 @@ def __missing__(self, key):
         self._borrowed_senders = {}
         self._borrow_sender_lock = asyncio.Lock()
 
+        self._loop = None  # only used as a sanity check
         self._updates_error = None
         self._updates_handle = None
         self._keepalive_handle = None
@@ -535,6 +536,11 @@ async def connect(self: 'TelegramClient') -> None:
         if self.session is None:
             raise ValueError('TelegramClient instance cannot be reused after logging out')
 
+        if self._loop is None:
+            self._loop = helpers.get_running_loop()
+        elif self._loop != helpers.get_running_loop():
+            raise RuntimeError('The asyncio event loop must not change after connection (see the FAQ for details)')
+
         if not await self._sender.connect(self._connection(
             self.session.server_address,
             self.session.port,

telethon/client/updates.py

@@ -319,6 +319,13 @@ async def _update_loop(self: 'TelegramClient'):
                             await self.disconnect()
                             break
                         continue
+                    except errors.TypeNotFoundError as e:
+                        # User is likely doing weird things with their account or session and Telegram gets confused as to what layer they use
+                        self._log[__name__].warning('Cannot get difference since the account is likely misusing the session: %s', e)
+                        self._message_box.end_difference()
+                        self._updates_error = e
+                        await self.disconnect()
+                        break
                     except OSError as e:
                         # Network is likely down, but it's unclear for how long.
                         # If disconnect is called this task will be cancelled along with the sleep.
@@ -354,6 +361,19 @@ async def _update_loop(self: 'TelegramClient'):
                             await self.disconnect()
                             break
                         continue
+                    except errors.TypeNotFoundError as e:
+                        self._log[__name__].warning(
+                            'Cannot get difference since the account is likely misusing the session: %s',
+                            get_diff.channel.channel_id, e
+                        )
+                        self._message_box.end_channel_difference(
+                            get_diff,
+                            PrematureEndReason.TEMPORARY_SERVER_ISSUES,
+                            self._mb_entity_cache
+                        )
+                        self._updates_error = e
+                        await self.disconnect()
+                        break
                     except (
                         errors.PersistentTimestampOutdatedError,
                         errors.PersistentTimestampInvalidError,

telethon/client/users.py

@@ -30,6 +30,10 @@ async def __call__(self: 'TelegramClient', request, ordered=False, flood_sleep_t
         return await self._call(self._sender, request, ordered=ordered)
 
     async def _call(self: 'TelegramClient', sender, request, ordered=False, flood_sleep_threshold=None):
+        if self._loop is not None and self._loop != helpers.get_running_loop():
+            raise RuntimeError('The asyncio event loop must not change after connection (see the FAQ for details)')
+        # if the loop is None it will fail with a connection error later on
+
         if flood_sleep_threshold is None:
             flood_sleep_threshold = self.flood_sleep_threshold
         requests = (request if utils.is_list_like(request) else (request,))
@@ -316,7 +320,9 @@ async def get_entity(
 
         # Merge users, chats and channels into a single dictionary
         id_entity = {
-            utils.get_peer_id(x): x
+            # `get_input_entity` might've guessed the type from a non-marked ID,
+            # so the only way to match that with the input is by not using marks here.
+            utils.get_peer_id(x, add_mark=False): x
             for x in itertools.chain(users, chats, channels)
         }
 
@@ -329,7 +335,7 @@ async def get_entity(
             if isinstance(x, str):
                 result.append(await self._get_entity_from_string(x))
             elif not isinstance(x, types.InputPeerSelf):
-                result.append(id_entity[utils.get_peer_id(x)])
+                result.append(id_entity[utils.get_peer_id(x, add_mark=False)])
             else:
                 result.append(next(
                     u for u in id_entity.values()

telethon/network/connection/connection.py

@@ -256,6 +256,8 @@ async def disconnect(self):
         if not self._connected:
             return
 
+        self._connected = False
+
         await helpers._cancel(
             self._log,
             send_task=self._send_task,
@@ -279,8 +281,6 @@ async def disconnect(self):
                     # * ConnectionResetError
                     self._log.info('%s during disconnect: %s', type(e), e)
 
-        self._connected = False
-
     def send(self, data):
         """
         Sends a packet of data through this connection mode.
@@ -333,6 +333,8 @@ async def _recv_loop(self):
             while self._connected:
                 try:
                     data = await self._recv()
+                except asyncio.CancelledError:
+                    break
                 except (IOError, asyncio.IncompleteReadError) as e:
                     self._log.warning('Server closed the connection: %s', e)
                     await self._recv_queue.put((None, e))
@@ -349,8 +351,6 @@ async def _recv_loop(self):
                     await self.disconnect()
                 else:
                     await self._recv_queue.put((data, None))
-        except asyncio.CancelledError:
-            pass
         finally:
             await self.disconnect()