Bump version to v4.3.1

Urllib3

.pre-commit-config.yaml

@@ -4,7 +4,7 @@
     -   id: yapf
         files: ^(telegram|tests)/.*\.py$
 -   repo: git://github.com/pre-commit/pre-commit-hooks
-    sha: adbb569fe9a64ad9bce3b53a77f1bc39ef31f682
+    sha: 6dfcb89af3c9b4d172cc2e5a8a2fa0f54615a338
     hooks:
     -   id: flake8
         files: ^telegram/.*\.py$

CHANGES.rst

@@ -1,3 +1,24 @@
+**2016-06-29**
+
+*Released 4.3.1*
+
+- Update wrong requirement: ``urllib3>=1.10``
+
+**2016-06-28**
+
+*Released 4.3*
+
+- Use ``urllib3.PoolManager`` for connection re-use
+- Rewrite ``run_async`` decorator to re-use threads
+- New requirements: ``urllib3`` and ``certifi``
+
+**2016-06-10**
+
+*Released 4.2.1*
+
+- Fix ``CallbackQuery.to_dict()`` bug (thanks to @jlmadurga)
+- Fix ``editMessageText`` exception when receiving a ``CallbackQuery``
+
 **2016-05-28**
 
 *Released 4.2*
@@ -45,7 +66,7 @@
 
 - Implement Bot API 2.0
 - Almost complete recode of ``Dispatcher``
-- Please read the `Transistion Guide to 4.0 <https://github.com/python-telegram-bot/python-telegram-bot/wiki/Transistion-guide-to-Version-4.0>`_
+- Please read the `Transition Guide to 4.0 <https://github.com/python-telegram-bot/python-telegram-bot/wiki/Transition-guide-to-Version-4.0>`_
 - **Changes from 4.0rc1**
     - The syntax of filters for ``MessageHandler`` (upper/lower cases)
     - Handler groups are now identified by ``int`` only, and ordered

README.rst

@@ -67,9 +67,9 @@ Table of contents
 
 - `License`_
 
-===============
-_`Introduction`
-===============
+============
+Introduction
+============
 
 This library provides a pure Python interface for the
 `Telegram Bot API <https://core.telegram.org/bots/api>`_.
@@ -81,15 +81,15 @@ In addition to the pure API implementation, this library features a number of hi
 make the development of bots easy and straightforward. These classes are contained in the
 ``telegram.ext`` submodule.
 
-=======================
-_`Telegram API support`
-=======================
+====================
+Telegram API support
+====================
 
 As of **28. May 2016**, all types and methods of the Telegram Bot API are supported.
 
-=============
-_`Installing`
-=============
+==========
+Installing
+==========
 
 You can install or upgrade python-telegram-bot with:
 
@@ -97,9 +97,9 @@ You can install or upgrade python-telegram-bot with:
 
     $ pip install python-telegram-bot --upgrade
 
-==================
-_`Getting started`
-==================
+===============
+Getting started
+===============
 
 Our Wiki contains a lot of resources to get you started with ``python-telegram-bot``:
 
@@ -111,9 +111,9 @@ Other references:
 - `Telegram API documentation <https://core.telegram.org/bots/api>`_
 - `python-telegram-bot documentation <https://pythonhosted.org/python-telegram-bot/>`_
 
-----------------------
-_`Learning by example`
-----------------------
+-------------------
+Learning by example
+-------------------
 
 We believe that the best way to learn and understand this simple package is by example. So here
 are some examples for you to review. Even if it's not your approach for learning, please take a
@@ -129,19 +129,13 @@ code and building on top of it.
 
 - `timerbot <https://github.com/python-telegram-bot/python-telegram-bot/blob/master/examples/timerbot.py>`_ uses the ``JobQueue`` to send timed messages.
 
-- `clibot <https://github.com/python-telegram-bot/python-telegram-bot/blob/master/examples/clibot.py>`_ has a command line interface.
-
-Examples using only the pure API:
-
-- `echobot <https://github.com/python-telegram-bot/python-telegram-bot/blob/master/examples/legacy/echobot.py>`_ replies back messages.
-
-- `roboed <https://github.com/python-telegram-bot/python-telegram-bot/blob/master/examples/legacy/roboed.py>`_ talks to `Robô Ed <http://www.ed.conpet.gov.br/br/converse.php>`_.
+- `echobot <https://github.com/python-telegram-bot/python-telegram-bot/blob/master/examples/legacy/echobot.py>`_ uses only the pure API to echo messages.
 
 Look at the examples on the `wiki <https://github.com/python-telegram-bot/python-telegram-bot/wiki/Examples>`_ to see other bots the community has built.
 
-----------
-_`Logging`
-----------
+-------
+Logging
+-------
 
 This library uses the ``logging`` module. To set up logging to standard output, put:
 
@@ -167,35 +161,35 @@ If you want DEBUG logs instead:
     logger.setLevel(logging.DEBUG)
 
 
-================
-_`Documentation`
-================
+=============
+Documentation
+=============
 
 ``python-telegram-bot``'s documentation lives at `pythonhosted.org <https://pythonhosted.org/python-telegram-bot/>`_.
 
-===============
-_`Getting help`
-===============
+============
+Getting help
+============
 
 You can get help in several ways:
 
 1. We have a vibrant community of developers helping each other in our `Telegram group <https://telegram.me/pythontelegrambotgroup>`_. Join us!
 
-2. Our `Wiki <https://github.com/python-telegram-bot/python-telegram-bot/wiki/>`_ offers a growing amount of resources.
+2. Our `Wiki pages <https://github.com/python-telegram-bot/python-telegram-bot/wiki/>`_ offer a growing amount of resources.
 
 3. You can ask for help on Stack Overflow using the `python-telegram-bot tag <https://stackoverflow.com/questions/tagged/python-telegram-bot>`_.
 
 4. As last resort, the developers are ready to help you with `serious issues <https://github.com/python-telegram-bot/python-telegram-bot/issues/new>`_.
 
 
-===============
-_`Contributing`
-===============
+============
+Contributing
+============
 
 Contributions of all sizes are welcome. Please review our `contribution guidelines <https://github.com/python-telegram-bot/python-telegram-bot/blob/master/.github/CONTRIBUTING.rst>`_ to get started. You can also help by `reporting bugs <https://github.com/python-telegram-bot/python-telegram-bot/issues/new>`_.
 
-==========
-_`License`
-==========
+=======
+License
+=======
 
-You may copy, distribute and modify the software provided that modifications are described and licensed for free under `LGPL-3 <https://www.gnu.org/licenses/lgpl-3.0.html>`_. Derivatives works (including modifications or anything statically linked to the library) can only be redistributed under `LGPL-3 <https://www.gnu.org/licenses/lgpl-3.0.html>`_, but applications that use the library don't have to be.
+You may copy, distribute and modify the software provided that modifications are described and licensed for free under `LGPL-3 <https://www.gnu.org/licenses/lgpl-3.0.html>`_. Derivatives works (including modifications or anything statically linked to the library) can only be redistributed under LGPL-3, but applications that use the library don't have to be.

examples/clibot.py

@@ -1,164 +0,0 @@
-#!/usr/bin/env python
-# -*- coding: utf-8 -*-
-#
-# Example Bot to show some of the functionality of the library
-# This program is dedicated to the public domain under the CC0 license.
-"""
-This Bot uses the Updater class to handle the bot.
-
-First, a few handler functions are defined. Then, those functions are passed to
-the Dispatcher and registered at their respective places.
-Then, the bot is started and the CLI-Loop is entered, where all text inputs are
-inserted into the update queue for the bot to handle.
-
-Usage:
-Repeats messages with a delay.
-Reply to last chat from the command line by typing "/reply <text>"
-Type 'stop' on the command line to stop the bot.
-"""
-
-from telegram.ext import Updater, StringCommandHandler, StringRegexHandler, \
-    MessageHandler, CommandHandler, RegexHandler, Filters
-from telegram.ext.dispatcher import run_async
-from time import sleep
-import logging
-
-from future.builtins import input
-
-# Enable Logging
-logging.basicConfig(format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
-                    level=logging.INFO)
-
-logger = logging.getLogger(__name__)
-
-# We use this var to save the last chat id, so we can reply to it
-last_chat_id = 0
-
-
-# Define a few (command) handler callback functions. These usually take the
-# two arguments bot and update. Error handlers also receive the raised
-# TelegramError object in error.
-def start(bot, update):
-    """ Answer in Telegram """
-    bot.sendMessage(update.message.chat_id, text='Hi!')
-
-
-def help(bot, update):
-    """ Answer in Telegram """
-    bot.sendMessage(update.message.chat_id, text='Help!')
-
-
-def any_message(bot, update):
-    """ Print to console """
-
-    # Save last chat_id to use in reply handler
-    global last_chat_id
-    last_chat_id = update.message.chat_id
-
-    logger.info("New message\nFrom: %s\nchat_id: %d\nText: %s" %
-                (update.message.from_user, update.message.chat_id, update.message.text))
-
-
-@run_async
-def message(bot, update):
-    """
-    Example for an asynchronous handler. It's not guaranteed that replies will
-    be in order when using @run_async. Also, you have to include **kwargs in
-    your parameter list. The kwargs contain all optional parameters that are
-    """
-
-    sleep(2)  # IO-heavy operation here
-    bot.sendMessage(update.message.chat_id, text='Echo: %s' % update.message.text)
-
-
-# These handlers are for updates of type str. We use them to react to inputs
-# on the command line interface
-def cli_reply(bot, update, args):
-    """
-    For any update of type telegram.Update or str that contains a command, you
-    can get the argument list by appending args to the function parameters.
-    Here, we reply to the last active chat with the text after the command.
-    """
-    if last_chat_id is not 0:
-        bot.sendMessage(chat_id=last_chat_id, text=' '.join(args))
-
-
-def cli_noncommand(bot, update, update_queue):
-    """
-    You can also get the update queue as an argument in any handler by
-    appending it to the argument list. Be careful with this though.
-    Here, we put the input string back into the queue, but as a command.
-
-    To learn more about those optional handler parameters, read the
-    documentation of the Handler classes.
-    """
-    update_queue.put('/%s' % update)
-
-
-def error(bot, update, error):
-    """ Print error to console """
-    logger.warn('Update %s caused error %s' % (update, error))
-
-
-def main():
-    # Create the EventHandler and pass it your bot's token.
-    token = 'TOKEN'
-    updater = Updater(token, workers=10)
-
-    # Get the dispatcher to register handlers
-    dp = updater.dispatcher
-
-    # This is how we add handlers for Telegram messages
-    dp.add_handler(CommandHandler("start", start))
-    dp.add_handler(CommandHandler("help", help))
-    # Message handlers only receive updates that don't contain commands
-    dp.add_handler(MessageHandler([Filters.text], message))
-    # Regex handlers will receive all updates on which their regex matches,
-    # but we have to add it in a separate group, since in one group,
-    # only one handler will be executed
-    dp.add_handler(RegexHandler('.*', any_message), group=1)
-
-    # String handlers work pretty much the same. Note that we have to tell
-    # the handler to pass the args or update_queue parameter
-    dp.add_handler(StringCommandHandler('reply', cli_reply, pass_args=True))
-    dp.add_handler(StringRegexHandler('[^/].*', cli_noncommand, pass_update_queue=True))
-
-    # All TelegramErrors are caught for you and delivered to the error
-    # handler(s). Other types of Errors are not caught.
-    dp.add_error_handler(error)
-
-    # Start the Bot and store the update Queue, so we can insert updates
-    update_queue = updater.start_polling(timeout=10)
-    '''
-    # Alternatively, run with webhook:
-
-    update_queue = updater.start_webhook('0.0.0.0',
-                                         443,
-                                         url_path=token,
-                                         cert='cert.pem',
-                                         key='key.key',
-                                         webhook_url='https://example.com/%s'
-                                             % token)
-
-    # Or, if SSL is handled by a reverse proxy, the webhook URL is already set
-    # and the reverse proxy is configured to deliver directly to port 6000:
-
-    update_queue = updater.start_webhook('0.0.0.0', 6000)
-    '''
-
-    # Start CLI-Loop
-    while True:
-        text = input()
-
-        # Gracefully stop the event handler
-        if text == 'stop':
-            updater.stop()
-            break
-
-        # else, put the text into the update queue to be handled by our handlers
-        elif len(text) > 0:
-            update_queue.put(text)
-
-
-if __name__ == '__main__':
-    main()

examples/echobot.py

@@ -1,16 +1,19 @@
 #!/usr/bin/env python
 # -*- coding: utf-8 -*-
 #
-# Simple Bot to reply to Telegram messages
+# Simple Bot to reply to Telegram messages. This is built on the API wrapper, see
+# echobot2.py to see the same example built on the telegram.ext bot framework.
 # This program is dedicated to the public domain under the CC0 license.
-
 import logging
 import telegram
 from telegram.error import NetworkError, Unauthorized
 from time import sleep
 
 
+update_id = None
+
 def main():
+    global update_id
     # Telegram Bot Authorization Token
     bot = telegram.Bot('TOKEN')
 
@@ -25,7 +28,7 @@ def main():
 
     while True:
         try:
-            update_id = echo(bot, update_id)
+            echo(bot)
         except NetworkError:
             sleep(1)
         except Unauthorized:
@@ -33,20 +36,17 @@ def main():
             update_id += 1
 
 
-def echo(bot, update_id):
-
+def echo(bot):
+    global update_id
     # Request updates after the last update_id
     for update in bot.getUpdates(offset=update_id, timeout=10):
         # chat_id is required to reply to any message
         chat_id = update.message.chat_id
         update_id = update.update_id + 1
-        message = update.message.text
 
-        if message:
+        if update.message:  # your bot can receive updates without messages
             # Reply to the message
-            bot.sendMessage(chat_id=chat_id, text=message)
-
-    return update_id
+            bot.sendMessage(chat_id=chat_id, text=update.message.text)
 
 
 if __name__ == '__main__':

examples/legacy/roboed.py

@@ -1,38 +0,0 @@
-#!/usr/bin/env python
-# encoding: utf-8
-#
-# Robô Ed Telegram Bot
-# This program is dedicated to the public domain under the CC0 license.
-
-import logging
-import telegram
-import urllib
-
-
-def main():
-    logging.basicConfig(format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')
-    bot = telegram.Bot('TOKEN')  # Telegram Bot Authorization Token
-
-    LAST_UPDATE_ID = bot.getUpdates()[-1].update_id  # Get lastest update
-
-    while True:
-        for update in bot.getUpdates(offset=LAST_UPDATE_ID, timeout=10):
-            text = update.message.text
-            chat_id = update.message.chat.id
-            update_id = update.update_id
-
-            if text:
-                roboed = ed(text)  # Ask something to Robô Ed
-                bot.sendMessage(chat_id=chat_id, text=roboed)
-                LAST_UPDATE_ID = update_id + 1
-
-
-def ed(text):
-    url = 'http://www.ed.conpet.gov.br/mod_perl/bot_gateway.cgi?server=0.0.0.0%3A8085&charset_post=utf-8&charset=utf-8&pure=1&js=0&tst=1&msg=' + text
-    data = urllib.urlopen(url).read()
-
-    return data.strip()
-
-
-if __name__ == '__main__':
-    main()

requirements.txt

@@ -1 +1,3 @@
-future
+future>=0.15.2
+urllib3>=1.10
+certifi

telegram/__init__.py

@@ -82,7 +82,7 @@ from .update import Update
 from .bot import Bot
 
 __author__ = 'devs@python-telegram-bot.org'
-__version__ = '4.2.0'
+__version__ = '4.3.1'
 __all__ = ['Audio', 'Bot', 'Chat', 'ChatMember', 'ChatAction', 'ChosenInlineResult',
            'CallbackQuery', 'Contact', 'Document', 'Emoji', 'File', 'ForceReply',
            'InlineKeyboardButton', 'InlineKeyboardMarkup', 'InlineQuery', 'InlineQueryResult',

telegram/bot.py

@@ -1009,6 +1009,7 @@ class Bot(TelegramObject):
         return result
 
     @log
+    @message
     def editMessageText(self,
                         text,
                         chat_id=None,
@@ -1016,7 +1017,6 @@ class Bot(TelegramObject):
                         inline_message_id=None,
                         parse_mode=None,
                         disable_web_page_preview=None,
-                        reply_markup=None,
                         **kwargs):
         """Use this method to edit text messages sent by the bot or via the bot
         (for inline bots).
@@ -1043,6 +1043,10 @@ class Bot(TelegramObject):
             A JSON-serialized object for an inline keyboard.
 
         Keyword Args:
+            reply_markup (Optional[:class:`telegram.ReplyMarkup`]): Additional
+                interface options. A JSON-serialized object for an inline
+                keyboard, custom reply keyboard, instructions to hide reply
+                keyboard or to force a reply from the user.
             timeout (Optional[float]): If this value is specified, use it as
                 the definitive timeout (in seconds) for urlopen() operations.
 
@@ -1070,15 +1074,8 @@ class Bot(TelegramObject):
             data['parse_mode'] = parse_mode
         if disable_web_page_preview:
             data['disable_web_page_preview'] = disable_web_page_preview
-        if reply_markup:
-            if isinstance(reply_markup, ReplyMarkup):
-                data['reply_markup'] = reply_markup.to_json()
-            else:
-                data['reply_markup'] = reply_markup
 
-        result = request.post(url, data, timeout=kwargs.get('timeout'))
-
-        return Message.de_json(result)
+        return url, data
 
     @log
     @message

telegram/callbackquery.py

@@ -43,3 +43,14 @@ class CallbackQuery(TelegramObject):
         data['message'] = Message.de_json(data.get('message'))
 
         return CallbackQuery(**data)
+
+    def to_dict(self):
+        """
+        Returns:
+            dict:
+        """
+        data = super(CallbackQuery, self).to_dict()
+
+        # Required
+        data['from'] = data.pop('from_user', None)
+        return data

telegram/ext/dispatcher.py

@@ -20,24 +20,47 @@
 
 import logging
 from functools import wraps
-from threading import Thread, BoundedSemaphore, Lock, Event, current_thread
+from threading import Thread, Lock, Event, current_thread
 from time import sleep
+from queue import Queue, Empty
 
-from queue import Empty
+from future.builtins import range
 
 from telegram import (TelegramError, NullHandler)
+from telegram.utils import request
 from telegram.ext.handler import Handler
 from telegram.utils.deprecate import deprecate
 
 logging.getLogger(__name__).addHandler(NullHandler())
 
-semaphore = None
-async_threads = set()
+ASYNC_QUEUE = Queue()
+ASYNC_THREADS = set()
 """:type: set[Thread]"""
-async_lock = Lock()
+ASYNC_LOCK = Lock()  # guards ASYNC_THREADS
 DEFAULT_GROUP = 0
 
 
+def _pooled():
+    """
+    A wrapper to run a thread in a thread pool
+    """
+    while 1:
+        try:
+            func, args, kwargs = ASYNC_QUEUE.get()
+
+        # If unpacking fails, the thread pool is being closed from Updater._join_async_threads
+        except TypeError:
+            logging.getLogger(__name__).debug("Closing run_async thread %s/%d" %
+                                              (current_thread().getName(), len(ASYNC_THREADS)))
+            break
+
+        try:
+            func(*args, **kwargs)
+
+        except:
+            logging.getLogger(__name__).exception("run_async function raised exception")
+
+
 def run_async(func):
     """
     Function decorator that will run the function in a new thread.
@@ -53,30 +76,11 @@ def run_async(func):
     #       set a threading.Event to notify caller thread
 
     @wraps(func)
-    def pooled(*pargs, **kwargs):
-        """
-        A wrapper to run a thread in a thread pool
-        """
-        try:
-            result = func(*pargs, **kwargs)
-        finally:
-            semaphore.release()
-
-            with async_lock:
-                async_threads.remove(current_thread())
-        return result
-
-    @wraps(func)
-    def async_func(*pargs, **kwargs):
+    def async_func(*args, **kwargs):
         """
         A wrapper to run a function in a thread
         """
-        thread = Thread(target=pooled, args=pargs, kwargs=kwargs)
-        semaphore.acquire()
-        with async_lock:
-            async_threads.add(thread)
-        thread.start()
-        return thread
+        ASYNC_QUEUE.put((func, args, kwargs))
 
     return async_func
 
@@ -107,11 +111,18 @@ class Dispatcher(object):
         self.__stop_event = Event()
         self.__exception_event = exception_event or Event()
 
-        global semaphore
-        if not semaphore:
-            semaphore = BoundedSemaphore(value=workers)
-        else:
-            self.logger.debug('Semaphore already initialized, skipping.')
+        with ASYNC_LOCK:
+            if not ASYNC_THREADS:
+                if request.is_con_pool_initialized():
+                    raise RuntimeError('Connection Pool already initialized')
+
+                request.CON_POOL_SIZE = workers + 3
+                for i in range(workers):
+                    thread = Thread(target=_pooled, name=str(i))
+                    ASYNC_THREADS.add(thread)
+                    thread.start()
+            else:
+                self.logger.debug('Thread pool already initialized, skipping.')
 
     def start(self):
         """
@@ -131,7 +142,7 @@ class Dispatcher(object):
         self.running = True
         self.logger.debug('Dispatcher started')
 
-        while True:
+        while 1:
             try:
                 # Pop update from update queue.
                 update = self.update_queue.get(True, 1)
@@ -145,7 +156,7 @@ class Dispatcher(object):
                 continue
 
             self.logger.debug('Processing Update: %s' % update)
-            self.processUpdate(update)
+            self.process_update(update)
 
         self.running = False
         self.logger.debug('Dispatcher thread stopped')
@@ -160,7 +171,7 @@ class Dispatcher(object):
                 sleep(0.1)
             self.__stop_event.clear()
 
-    def processUpdate(self, update):
+    def process_update(self, update):
         """
         Processes a single update.
 
@@ -170,7 +181,7 @@ class Dispatcher(object):
 
         # An error happened while polling
         if isinstance(update, TelegramError):
-            self.dispatchError(None, update)
+            self.dispatch_error(None, update)
 
         else:
             for group in self.groups:
@@ -185,7 +196,7 @@ class Dispatcher(object):
                                          'Update.')
 
                         try:
-                            self.dispatchError(update, te)
+                            self.dispatch_error(update, te)
                         except Exception:
                             self.logger.exception('An uncaught error was raised while '
                                                   'handling the error')
@@ -271,7 +282,7 @@ class Dispatcher(object):
         if callback in self.error_handlers:
             self.error_handlers.remove(callback)
 
-    def dispatchError(self, update, error):
+    def dispatch_error(self, update, error):
         """
         Dispatches an error.
 

telegram/ext/updater.py

@@ -309,7 +309,7 @@ class Updater(object):
 
     def _bootstrap(self, max_retries, clean, webhook_url, cert=None):
         retries = 0
-        while True:
+        while 1:
 
             try:
                 if clean:
@@ -346,7 +346,7 @@ class Updater(object):
 
         self.job_queue.stop()
         with self.__lock:
-            if self.running:
+            if self.running or dispatcher.ASYNC_THREADS:
                 self.logger.debug('Stopping Updater and Dispatcher...')
 
                 self.running = False
@@ -354,9 +354,8 @@ class Updater(object):
                 self._stop_httpd()
                 self._stop_dispatcher()
                 self._join_threads()
-                # async threads must be join()ed only after the dispatcher
-                # thread was joined, otherwise we can still have new async
-                # threads dispatched
+                # async threads must be join()ed only after the dispatcher thread was joined,
+                # otherwise we can still have new async threads dispatched
                 self._join_async_threads()
 
     def _stop_httpd(self):
@@ -372,13 +371,19 @@ class Updater(object):
         self.dispatcher.stop()
 
     def _join_async_threads(self):
-        with dispatcher.async_lock:
-            threads = list(dispatcher.async_threads)
-        total = len(threads)
-        for i, thr in enumerate(threads):
-            self.logger.debug('Waiting for async thread {0}/{1} to end'.format(i, total))
-            thr.join()
-            self.logger.debug('async thread {0}/{1} has ended'.format(i, total))
+        with dispatcher.ASYNC_LOCK:
+            threads = list(dispatcher.ASYNC_THREADS)
+            total = len(threads)
+
+            # Stop all threads in the thread pool by put()ting one non-tuple per thread
+            for i in range(total):
+                dispatcher.ASYNC_QUEUE.put(None)
+
+            for i, thr in enumerate(threads):
+                self.logger.debug('Waiting for async thread {0}/{1} to end'.format(i + 1, total))
+                thr.join()
+                dispatcher.ASYNC_THREADS.remove(thr)
+                self.logger.debug('async thread {0}/{1} has ended'.format(i + 1, total))
 
     def _join_threads(self):
         for thr in self.__threads:

telegram/utils/request.py

@@ -18,29 +18,56 @@
 # along with this program.  If not, see [http://www.gnu.org/licenses/].
 """This module contains methods to make POST and GET requests"""
 
-import functools
 import json
 import socket
-from ssl import SSLError
+import logging
 
-from future.moves.http.client import HTTPException
-from future.moves.urllib.error import HTTPError, URLError
-from future.moves.urllib.request import urlopen, urlretrieve, Request
+import certifi
+import urllib3
+from urllib3.connection import HTTPConnection
 
 from telegram import (InputFile, TelegramError)
 from telegram.error import Unauthorized, NetworkError, TimedOut, BadRequest
 
+_CON_POOL = None
+""":type: urllib3.PoolManager"""
+CON_POOL_SIZE = 1
+
+logging.getLogger('urllib3').setLevel(logging.WARNING)
+
+
+def _get_con_pool():
+    global _CON_POOL
+
+    if _CON_POOL is not None:
+        return _CON_POOL
+
+    _CON_POOL = urllib3.PoolManager(maxsize=CON_POOL_SIZE,
+                                    cert_reqs='CERT_REQUIRED',
+                                    ca_certs=certifi.where(),
+                                    socket_options=HTTPConnection.default_socket_options + [
+                                        (socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1),
+                                    ])
+    return _CON_POOL
+
+
+def is_con_pool_initialized():
+    return _CON_POOL is not None
+
+
+def stop_con_pool():
+    global _CON_POOL
+    if _CON_POOL is not None:
+        _CON_POOL.clear()
+        _CON_POOL = None
+
 
 def _parse(json_data):
-    """Try and parse the JSON returned from Telegram and return an empty
-    dictionary if there is any error.
-
-    Args:
-      url:
-        urllib.urlopen object
+    """Try and parse the JSON returned from Telegram.
 
     Returns:
-      A JSON parsed as Python dict with results.
+        dict: A JSON parsed as Python dict with results - on error this dict will be empty.
+
     """
     decoded_s = json_data.decode('utf-8')
     try:
@@ -54,53 +81,49 @@ def _parse(json_data):
     return data['result']
 
 
-def _try_except_req(func):
-    """Decorator for requests to handle known exceptions"""
+def _request_wrapper(*args, **kwargs):
+    """Wraps urllib3 request for handling known exceptions.
 
-    @functools.wraps(func)
-    def decorator(*args, **kwargs):
-        try:
-            return func(*args, **kwargs)
+    Args:
+        args: unnamed arguments, passed to urllib3 request.
+        kwargs: keyword arguments, passed tp urllib3 request.
 
-        except HTTPError as error:
-            # `HTTPError` inherits from `URLError` so `HTTPError` handling must
-            # come first.
-            errcode = error.getcode()
+    Returns:
+        str: A non-parsed JSON text.
 
-            try:
-                message = _parse(error.read())
+    Raises:
+        TelegramError
 
-                if errcode in (401, 403):
-                    raise Unauthorized()
-                elif errcode == 400:
-                    raise BadRequest(message)
-                elif errcode == 502:
-                    raise NetworkError('Bad Gateway')
-            except ValueError:
-                message = 'Unknown HTTPError {0}'.format(error.getcode())
+    """
 
-            raise NetworkError('{0} ({1})'.format(message, errcode))
+    try:
+        resp = _get_con_pool().request(*args, **kwargs)
+    except urllib3.exceptions.TimeoutError as error:
+        raise TimedOut()
+    except urllib3.exceptions.HTTPError as error:
+        # HTTPError must come last as its the base urllib3 exception class
+        # TODO: do something smart here; for now just raise NetworkError
+        raise NetworkError('urllib3 HTTPError {0}'.format(error))
 
-        except URLError as error:
-            raise NetworkError('URLError: {0}'.format(error.reason))
+    if 200 <= resp.status <= 299:
+        # 200-299 range are HTTP success statuses
+        return resp.data
 
-        except (SSLError, socket.timeout) as error:
-            err_s = str(error)
-            if 'operation timed out' in err_s:
-                raise TimedOut()
+    try:
+        message = _parse(resp.data)
+    except ValueError:
+        raise NetworkError('Unknown HTTPError {0}'.format(resp.status))
 
-            raise NetworkError(err_s)
-
-        except HTTPException as error:
-            raise NetworkError('HTTPException: {0!r}'.format(error))
-
-        except socket.error as error:
-            raise NetworkError('socket.error: {0!r}'.format(error))
-
-    return decorator
+    if resp.status in (401, 403):
+        raise Unauthorized()
+    elif resp.status == 400:
+        raise BadRequest(repr(message))
+    elif resp.status == 502:
+        raise NetworkError('Bad Gateway')
+    else:
+        raise NetworkError('{0} ({1})'.format(message, resp.status))
 
 
-@_try_except_req
 def get(url):
     """Request an URL.
     Args:
@@ -109,13 +132,13 @@ def get(url):
 
     Returns:
       A JSON object.
+
     """
-    result = urlopen(url).read()
+    result = _request_wrapper('GET', url)
 
     return _parse(result)
 
 
-@_try_except_req
 def post(url, data, timeout=None):
     """Request an URL.
     Args:
@@ -142,16 +165,17 @@ def post(url, data, timeout=None):
 
     if InputFile.is_inputfile(data):
         data = InputFile(data)
-        request = Request(url, data=data.to_form(), headers=data.headers)
+        result = _request_wrapper('POST', url, body=data.to_form(), headers=data.headers)
     else:
         data = json.dumps(data)
-        request = Request(url, data=data.encode(), headers={'Content-Type': 'application/json'})
+        result = _request_wrapper('POST',
+                                  url,
+                                  body=data.encode(),
+                                  headers={'Content-Type': 'application/json'})
 
-    result = urlopen(request, **urlopen_kwargs).read()
     return _parse(result)
 
 
-@_try_except_req
 def download(url, filename):
     """Download a file by its URL.
     Args:
@@ -160,6 +184,8 @@ def download(url, filename):
 
       filename:
         The filename within the path to download the file.
-    """
 
-    urlretrieve(url, filename)
+    """
+    buf = _request_wrapper('GET', url)
+    with open(filename, 'wb') as fobj:
+        fobj.write(buf)

tests/test_bot.py

@@ -20,6 +20,7 @@
 """This module contains a object that represents Tests for Telegram Bot"""
 
 import io
+import re
 from datetime import datetime
 import sys
 
@@ -211,10 +212,11 @@ class BotTest(BaseTest, unittest.TestCase):
     @flaky(3, 1)
     @timeout(10)
     def testLeaveChat(self):
-        with self.assertRaisesRegexp(telegram.error.BadRequest, 'Chat not found'):
+        regex = re.compile('chat not found', re.IGNORECASE)
+        with self.assertRaisesRegexp(telegram.error.BadRequest, regex):
             chat = self._bot.leaveChat(-123456)
 
-        with self.assertRaisesRegexp(telegram.error.NetworkError, 'Chat not found'):
+        with self.assertRaisesRegexp(telegram.error.NetworkError, regex):
             chat = self._bot.leaveChat(-123456)
 
     @flaky(3, 1)

tests/test_jobqueue.py

@@ -31,6 +31,7 @@ else:
 
 sys.path.append('.')
 
+from telegram.utils.request import stop_con_pool
 from telegram.ext import JobQueue, Updater
 from tests.base import BaseTest
 
@@ -58,6 +59,7 @@ class JobQueueTest(BaseTest, unittest.TestCase):
     def tearDown(self):
         if self.jq is not None:
             self.jq.stop()
+        stop_con_pool()
 
     def job1(self, bot):
         self.result += 1

tests/test_photo.py

@@ -35,15 +35,14 @@ class PhotoTest(BaseTest, unittest.TestCase):
 
     def setUp(self):
         self.photo_file = open('tests/data/telegram.jpg', 'rb')
-        self.photo_file_id = 'AgADAQADvb8xGx8j9QcpZDKxYoFK3bfX1i8ABFX_dgMWoKDuQugAAgI'
+        self.photo_file_id = 'AgADAQADgEsyGx8j9QfmDMmwkPBrFcKRzy8ABHW8ul9nW7FoNHYBAAEC'
         self.photo_file_url = 'https://raw.githubusercontent.com/python-telegram-bot/python-telegram-bot/master/tests/data/telegram.jpg'
         self.width = 300
         self.height = 300
         self.thumb = {
             'width': 90,
             'height': 90,
-            'file_id':
-            'AgADAQADvb8xGx8j9QcpZDKxYoFK3bfX1i8ABBxRLXFhLnhIQ-gAAgI',
+            'file_id': 'AgADAQADgEsyGx8j9QfmDMmwkPBrFcKRzy8ABD64nkFkjujeNXYBAAEC',
             'file_size': 1478
         }
         self.file_size = 10209

tests/test_updater.py

@@ -48,6 +48,7 @@ except ImportError:
 sys.path.append('.')
 
 from telegram import Update, Message, TelegramError, User, Chat, Bot
+from telegram.utils.request import stop_con_pool
 from telegram.ext import *
 from telegram.ext.dispatcher import run_async
 from telegram.error import Unauthorized, InvalidToken
@@ -78,12 +79,14 @@ class UpdaterTest(BaseTest, unittest.TestCase):
         self.lock = Lock()
 
     def _setup_updater(self, *args, **kwargs):
+        stop_con_pool()
         bot = MockBot(*args, **kwargs)
         self.updater = Updater(workers=2, bot=bot)
 
     def tearDown(self):
         if self.updater is not None:
             self.updater.stop()
+        stop_con_pool()
 
     def reset(self):
         self.message_count = 0
@@ -639,8 +642,8 @@ class UpdaterTest(BaseTest, unittest.TestCase):
         self.assertFalse(self.updater.running)
 
     def test_createBot(self):
-        updater = Updater('123:abcd')
-        self.assertIsNotNone(updater.bot)
+        self.updater = Updater('123:abcd')
+        self.assertIsNotNone(self.updater.bot)
 
     def test_mutualExclusiveTokenBot(self):
         bot = Bot('123:zyxw')