add 'user_agent' config for the regex source

This commit is contained in:
lilydjwg 2015-03-14 15:54:26 +08:00
parent ca931633aa
commit e4830dd162
2 changed files with 5 additions and 0 deletions

View File

@ -91,6 +91,9 @@ regex
proxy proxy
The HTTP proxy to use. The format is ``host:port``, e.g. ``localhost:8087``. This requires `pycurl <http://pycurl.sourceforge.net/>`_. The HTTP proxy to use. The format is ``host:port``, e.g. ``localhost:8087``. This requires `pycurl <http://pycurl.sourceforge.net/>`_.
user_agent
The ``User-Agent`` header value to use. Use something more like a tool (e.g. ``curl/7.40.0``) in Europe or the real web page won't get through because cookie policies (SourceForge has this issue).
Find with a Command Find with a Command
------------------- -------------------
Use a shell command line to get the version. The output is striped first, so trailing newlines do not bother. Use a shell command line to get the version. The output is striped first, so trailing newlines do not bother.

View File

@ -30,6 +30,8 @@ def get_version(name, conf, callback):
kwargs['proxy_port'] = int(port) kwargs['proxy_port'] = int(port)
else: else:
logger.warn('%s: proxy set but not used because pycurl is unavailable.', name) logger.warn('%s: proxy set but not used because pycurl is unavailable.', name)
if conf.get('user_agent'):
kwargs['user_agent'] = conf['user_agent']
httpclient.fetch(conf['url'], partial( httpclient.fetch(conf['url'], partial(
_got_version, name, r, encoding, callback _got_version, name, r, encoding, callback