mirror of
https://github.com/lilydjwg/nvchecker
synced 2025-02-02 11:31:50 +00:00
add documentation for plugins
This commit is contained in:
lilydjwg
parent
0eb5860584
commit
8c99ae8eea
146
README.rst
146
README.rst
@ -1,3 +1,6 @@
|
||||
.. contents::
|
||||
:local:
|
||||
|
||||
**nvchecker** (short for *new version checker*) is for checking if a new version of some software has been released.
|
||||
|
||||
This is the version 2.0 branch that is in beta. Things may still change without backward-compatibility.
|
||||
@ -17,53 +20,8 @@ For the old version 1.x, please switch to the ``v1.x`` branch.
|
||||
:alt: Packaging status
|
||||
:target: https://repology.org/metapackage/nvchecker/versions
|
||||
|
||||
Contents
|
||||
========
|
||||
|
||||
* `Dependency <#dependency>`_
|
||||
* `Install and Run <#install-and-run>`_
|
||||
|
||||
* `JSON logging <#json-logging>`_
|
||||
* `Upgrade from 1.x version <#upgrade-from-1x-version>`_
|
||||
|
||||
* `Version Record Files <#version-record-files>`_
|
||||
|
||||
* `The nvtake Command <#the-nvtake-command>`_
|
||||
|
||||
* `Configuration Files <#configuration-files>`_
|
||||
|
||||
* `Configuration Table <#configuration-table>`_
|
||||
* `Global Options <#global-options>`_
|
||||
* `List Options <#list-options>`_
|
||||
* `Search in a Webpage <#search-in-a-webpage>`_
|
||||
* `Find with a Command <#find-with-a-command>`_
|
||||
* `Check AUR <#check-aur>`_
|
||||
* `Check GitHub <#check-github>`_
|
||||
* `Check BitBucket <#check-bitbucket>`_
|
||||
* `Check GitLab <#check-gitlab>`_
|
||||
* `Check Gitea <#check-gitea>`_
|
||||
* `Check PyPI <#check-pypi>`_
|
||||
* `Check RubyGems <#check-rubygems>`_
|
||||
* `Check NPM Registry <#check-npm-registry>`_
|
||||
* `Check Hackage <#check-hackage>`_
|
||||
* `Check CPAN <#check-cpan>`_
|
||||
* `Check Packagist <#check-packagist>`_
|
||||
* `Check Local Pacman Database <#check-local-pacman-database>`_
|
||||
* `Check Arch Linux official packages <#check-arch-linux-official-packages>`_
|
||||
* `Check Debian Linux official packages <#check-debian-linux-official-packages>`_
|
||||
* `Check Ubuntu Linux official packages <#check-ubuntu-linux-official-packages>`_
|
||||
* `Check Repology (repology.org) <#check-repology>`_
|
||||
* `Check Anitya (release-monitoring.org) <#check-anitya>`_
|
||||
* `Check Android SDK <#check-android-sdk>`_
|
||||
* `Check Sparkle framework <#check-sparkle-framework>`_
|
||||
* `Check Pagure <#check-pagure>`_
|
||||
* `Manually updating <#manually-updating>`_
|
||||
* `Extending <#extending>`_
|
||||
|
||||
* `Bugs <#bugs>`_
|
||||
|
||||
Dependency
|
||||
==========
|
||||
----------
|
||||
- Python 3.7+
|
||||
- Python library: structlog, toml, appdirs
|
||||
- One of these Python library combinations (ordered by preference):
|
||||
@ -76,7 +34,7 @@ Dependency
|
||||
- All commands used in your software version configuration files
|
||||
|
||||
Install and Run
|
||||
===============
|
||||
---------------
|
||||
To install::
|
||||
|
||||
pip3 install nvchecker
|
||||
@ -96,7 +54,7 @@ Run with one or more software version files::
|
||||
You normally will like to specify some "version record files"; see below.
|
||||
|
||||
JSON logging
|
||||
------------
|
||||
~~~~~~~~~~~~
|
||||
With ``--logger=json`` or ``--logger=both``, you can get a structured logging
|
||||
for programmatically consuming. You can use ``--json-log-fd=FD`` to specify the
|
||||
file descriptor to send logs to (take care to do line buffering). The logging
|
||||
@ -120,7 +78,7 @@ level=error
|
||||
further information.
|
||||
|
||||
Upgrade from 1.x version
|
||||
------------------------
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
There are several backward-incompatible changes from the previous 1.x version.
|
||||
|
||||
@ -131,14 +89,14 @@ There are several backward-incompatible changes from the previous 1.x version.
|
||||
5. All software configuration tables need a ``source`` option to specify which source is to be used rather than being figured out from option names in use. This enables additional source plugins to be discovered.
|
||||
6. The version record files have been changed to use JSON format (the old format will be converted on writing).
|
||||
7. The ``vcs`` source is removed. (It's available inside `lilac <https://github.com/archlinuxcn/lilac>`_ at the moment.)
|
||||
8. ``include_tags_pattern`` and ``ignored_tags`` are removed. Use `list options`_ instead.
|
||||
8. ``include_tags_pattern`` and ``ignored_tags`` are removed. Use :ref:`list options` instead.
|
||||
|
||||
Version Record Files
|
||||
====================
|
||||
--------------------
|
||||
Version record files record which version of the software you know or is available. They are a simple JSON object mapping software names to known versions.
|
||||
|
||||
The ``nvtake`` Command
|
||||
----------------------
|
||||
~~~~~~~~~~~~~~~~~~~~~~
|
||||
This command helps to manage version record files. It reads both old and new version record files, and a list of names given on the commandline. It then update the versions of those names in the old version record file.
|
||||
|
||||
This helps when you have known (and processed) some of the updated software, but not all. You can tell nvchecker that via this command instead of editing the file by hand.
|
||||
@ -146,7 +104,7 @@ This helps when you have known (and processed) some of the updated software, but
|
||||
This command will help most if you specify where you version record files are in your config file. See below for how to use a config file.
|
||||
|
||||
The ``nvcmp`` Command
|
||||
----------------------
|
||||
~~~~~~~~~~~~~~~~~~~~~
|
||||
This command compares the ``newver`` file with the ``oldver`` one and prints out any differences as updates, e.g.::
|
||||
|
||||
$ nvcmp -c sample_source.toml
|
||||
@ -154,13 +112,13 @@ This command compares the ``newver`` file with the ``oldver`` one and prints out
|
||||
test 0.0 -> 0.1
|
||||
|
||||
Configuration Files
|
||||
===================
|
||||
-------------------
|
||||
The software version source files are in `toml`_ format. The *key name* is the name of the software. Following fields are used to tell nvchecker how to determine the current version of that software.
|
||||
|
||||
See ``sample_source.toml`` for an example.
|
||||
|
||||
Configuration Table
|
||||
-------------------
|
||||
~~~~~~~~~~~~~~~~~~~
|
||||
A special table named ``__config__`` provides some configuration options.
|
||||
|
||||
Relative path are relative to the source files, and ``~`` and environmental variables are expanded.
|
||||
@ -185,7 +143,7 @@ keyfile
|
||||
specific source for the key name(s) to use.
|
||||
|
||||
Global Options
|
||||
--------------
|
||||
~~~~~~~~~~~~~~
|
||||
The following options apply to all check sources.
|
||||
|
||||
prefix
|
||||
@ -211,6 +169,9 @@ proxy
|
||||
and doesn't work with the ``aur`` source because it's batched. However the
|
||||
global proxy config applies.
|
||||
|
||||
user_agent
|
||||
The user agent string to use for HTTP requests.
|
||||
|
||||
tries
|
||||
Try specified times when a network error occurs. Default is ``1``.
|
||||
|
||||
@ -222,8 +183,10 @@ and then do something special, just use ``from_pattern```/``to_pattern``. For
|
||||
example, the transformation of ``v1_1_0`` => ``1.1.0`` can be achieved with
|
||||
``from_pattern = v(\d+)_(\d+)_(\d+)`` and ``to_pattern = \1.\2.\3``.
|
||||
|
||||
.. _list options:
|
||||
|
||||
List Options
|
||||
------------
|
||||
~~~~~~~~~~~~
|
||||
|
||||
The following options apply to sources that return a list. See
|
||||
individual source tables to determine whether they are
|
||||
@ -251,7 +214,7 @@ ignored
|
||||
"overridden" by the old broken ones.
|
||||
|
||||
Search in a Webpage
|
||||
-------------------
|
||||
~~~~~~~~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "regex"
|
||||
@ -271,10 +234,10 @@ regex
|
||||
|
||||
When multiple version strings are found, the maximum of those is chosen.
|
||||
|
||||
This source supports `list options`_.
|
||||
This source supports :ref:`list options`.
|
||||
|
||||
Find with a Command
|
||||
-------------------
|
||||
~~~~~~~~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "cmd"
|
||||
@ -285,7 +248,7 @@ cmd
|
||||
The command line to use. This will run with the system's standard shell (i.e. ``/bin/sh``).
|
||||
|
||||
Check AUR
|
||||
---------
|
||||
~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "aur"
|
||||
@ -304,7 +267,7 @@ use_last_modified
|
||||
Append last modified time to the version.
|
||||
|
||||
Check GitHub
|
||||
------------
|
||||
~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "github"
|
||||
@ -359,10 +322,10 @@ To set an authorization token, you can set:
|
||||
- a key named ``github`` in the keyfile
|
||||
- the token option
|
||||
|
||||
This source supports `list options`_ when ``use_max_tag`` is set.
|
||||
This source supports :ref:`list options` when ``use_max_tag`` is set.
|
||||
|
||||
Check Gitea
|
||||
-------------
|
||||
~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "gitea"
|
||||
@ -391,10 +354,10 @@ To set an authorization token, you can set:
|
||||
- a key named ``gitea_{host}`` in the keyfile, where ``host`` is all-lowercased host name
|
||||
- the token option
|
||||
|
||||
This source supports `list options`_ when ``use_max_tag`` is set.
|
||||
This source supports :ref:`list options` when ``use_max_tag`` is set.
|
||||
|
||||
Check BitBucket
|
||||
---------------
|
||||
~~~~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "bitbucket"
|
||||
@ -416,10 +379,10 @@ max_page
|
||||
How many pages do we search for the max tag? Default is 3. This works when
|
||||
``use_max_tag`` is set.
|
||||
|
||||
This source supports `list options`_ when ``use_max_tag`` is set.
|
||||
This source supports :ref:`list options` when ``use_max_tag`` is set.
|
||||
|
||||
Check GitLab
|
||||
-------------
|
||||
~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "gitlab"
|
||||
@ -448,10 +411,10 @@ To set an authorization token, you can set:
|
||||
- a key named ``gitlab_{host}`` in the keyfile, where ``host`` is all-lowercased host name
|
||||
- the token option
|
||||
|
||||
This source supports `list options`_ when ``use_max_tag`` is set.
|
||||
This source supports :ref:`list options` when ``use_max_tag`` is set.
|
||||
|
||||
Check PyPI
|
||||
----------
|
||||
~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "pypi"
|
||||
@ -465,7 +428,7 @@ use_pre_release
|
||||
Whether to accept pre release. Default is false.
|
||||
|
||||
Check RubyGems
|
||||
--------------
|
||||
~~~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "gems"
|
||||
@ -476,7 +439,7 @@ gems
|
||||
The name used on RubyGems, e.g. ``sass``.
|
||||
|
||||
Check NPM Registry
|
||||
------------------
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "npm"
|
||||
@ -487,7 +450,7 @@ npm
|
||||
The name used on NPM Registry, e.g. ``coffee-script``.
|
||||
|
||||
Check Hackage
|
||||
-------------
|
||||
~~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "hackage"
|
||||
@ -498,7 +461,7 @@ hackage
|
||||
The name used on Hackage, e.g. ``pandoc``.
|
||||
|
||||
Check CPAN
|
||||
--------------
|
||||
~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "cpan"
|
||||
@ -509,7 +472,7 @@ cpan
|
||||
The name used on CPAN, e.g. ``YAML``.
|
||||
|
||||
Check Packagist
|
||||
---------------
|
||||
~~~~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "packagist"
|
||||
@ -520,7 +483,7 @@ packagist
|
||||
The name used on Packagist, e.g. ``monolog/monolog``.
|
||||
|
||||
Check Local Pacman Database
|
||||
---------------------------
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "pacman"
|
||||
@ -534,7 +497,7 @@ strip_release
|
||||
Strip the release part.
|
||||
|
||||
Check Arch Linux official packages
|
||||
----------------------------------
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "archpkg"
|
||||
@ -551,7 +514,7 @@ provided
|
||||
Instead of the package version, return the version this package provides. Its value is what the package provides, and ``strip-release`` takes effect too. This is best used with libraries.
|
||||
|
||||
Check Debian Linux official packages
|
||||
------------------------------------
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "debianpkg"
|
||||
@ -568,7 +531,7 @@ strip_release
|
||||
Strip the release part.
|
||||
|
||||
Check Ubuntu Linux official packages
|
||||
------------------------------------
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "ubuntupkg"
|
||||
@ -585,7 +548,7 @@ strip_release
|
||||
Strip the release part.
|
||||
|
||||
Check Repology
|
||||
--------------
|
||||
~~~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "repology"
|
||||
@ -599,7 +562,7 @@ repo
|
||||
Check the version in this repo. This field is required.
|
||||
|
||||
Check Anitya
|
||||
------------
|
||||
~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "anitya"
|
||||
@ -610,7 +573,7 @@ anitya
|
||||
``distro/package``, where ``distro`` can be a lot of things like "fedora", "arch linux", "gentoo", etc. ``package`` is the package name of the chosen distribution.
|
||||
|
||||
Check Android SDK
|
||||
-----------------
|
||||
~~~~~~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "android_sdk"
|
||||
@ -624,7 +587,7 @@ repo
|
||||
Should be one of ``addon`` or ``package``. Packages in ``addon2-1.xml`` use ``addon`` and packages in ``repository2-1.xml`` use ``package``.
|
||||
|
||||
Check Sparkle framework
|
||||
-----------------------
|
||||
~~~~~~~~~~~~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "sparkle"
|
||||
@ -635,7 +598,7 @@ sparkle
|
||||
The url of the sparkle appcast.
|
||||
|
||||
Check Pagure
|
||||
------------
|
||||
~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "pagure"
|
||||
@ -648,10 +611,10 @@ pagure
|
||||
host
|
||||
Hostname of alternative instance like src.fedoraproject.org.
|
||||
|
||||
This source returns tags and supports `list options`_.
|
||||
This source returns tags and supports :ref:`list options`.
|
||||
|
||||
Manually updating
|
||||
-----------------
|
||||
~~~~~~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "manual"
|
||||
@ -662,14 +625,9 @@ manual
|
||||
The version string.
|
||||
|
||||
Extending
|
||||
---------
|
||||
It's possible to extend the supported sources by writing plugins. See
|
||||
``plugins.rst`` for documentation.
|
||||
|
||||
Bugs
|
||||
====
|
||||
* Finish writing results even on Ctrl-C or other interruption.
|
||||
~~~~~~~~~
|
||||
It's possible to extend the supported sources by writing
|
||||
plugins. See :doc:`plugin` for documentation.
|
||||
|
||||
.. _Pacman: https://wiki.archlinux.org/index.php/Pacman
|
||||
.. _list options: #list-options
|
||||
.. _toml: https://toml.io/
|
||||
|
||||
1
docs/.gitignore
vendored
Normal file
1
docs/.gitignore
vendored
Normal file
@ -0,0 +1 @@
|
||||
_build/
|
||||
20
docs/Makefile
Normal file
20
docs/Makefile
Normal file
@ -0,0 +1,20 @@
|
||||
# Minimal makefile for Sphinx documentation
|
||||
#
|
||||
|
||||
# You can set these variables from the command line, and also
|
||||
# from the environment for the first two.
|
||||
SPHINXOPTS ?=
|
||||
SPHINXBUILD ?= sphinx-build
|
||||
SOURCEDIR = .
|
||||
BUILDDIR = _build
|
||||
|
||||
# Put it first so that "make" without argument is like "make help".
|
||||
help:
|
||||
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
|
||||
|
||||
.PHONY: help Makefile
|
||||
|
||||
# Catch-all target: route all unknown targets to Sphinx using the new
|
||||
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
|
||||
%: Makefile
|
||||
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
|
||||
20
docs/api.rst
Normal file
20
docs/api.rst
Normal file
@ -0,0 +1,20 @@
|
||||
``nvchecker.api`` --- The source plugin API
|
||||
===========================================
|
||||
|
||||
.. automodule:: nvchecker.api
|
||||
:members:
|
||||
:imported-members:
|
||||
:undoc-members:
|
||||
|
||||
.. py:data:: session
|
||||
:type: nvchecker.httpclient.base.BaseSession
|
||||
|
||||
The object to send out HTTP requests, respecting various options in the configuration entry.
|
||||
|
||||
.. automodule:: nvchecker.httpclient.base
|
||||
:members: BaseSession, Response
|
||||
:undoc-members:
|
||||
|
||||
.. autodata:: nvchecker.api.proxy
|
||||
.. autodata:: nvchecker.api.user_agent
|
||||
.. autodata:: nvchecker.api.tries
|
||||
48
docs/conf.py
Normal file
48
docs/conf.py
Normal file
@ -0,0 +1,48 @@
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.abspath(".."))
|
||||
import nvchecker
|
||||
|
||||
master_doc = "index"
|
||||
|
||||
project = "nvchecker"
|
||||
copyright = "lilydjwg, et al."
|
||||
|
||||
version = release = nvchecker.__version__
|
||||
|
||||
extensions = [
|
||||
"sphinx.ext.autodoc",
|
||||
"sphinx.ext.doctest",
|
||||
"sphinx.ext.intersphinx",
|
||||
"sphinx.ext.viewcode",
|
||||
]
|
||||
|
||||
primary_domain = "py"
|
||||
default_role = "py:obj"
|
||||
|
||||
autodoc_member_order = "bysource"
|
||||
autoclass_content = "both"
|
||||
autodoc_inherit_docstrings = False
|
||||
|
||||
# Without this line sphinx includes a copy of object.__init__'s docstring
|
||||
# on any class that doesn't define __init__.
|
||||
# https://bitbucket.org/birkenfeld/sphinx/issue/1337/autoclass_content-both-uses-object__init__
|
||||
autodoc_docstring_signature = False
|
||||
|
||||
intersphinx_mapping = {"python": ("https://docs.python.org/3.8/", None)}
|
||||
|
||||
on_rtd = os.environ.get("READTHEDOCS", None) == "True"
|
||||
|
||||
# On RTD we can't import sphinx_rtd_theme, but it will be applied by
|
||||
# default anyway. This block will use the same theme when building locally
|
||||
# as on RTD.
|
||||
if not on_rtd:
|
||||
import sphinx_rtd_theme
|
||||
|
||||
html_theme = "sphinx_rtd_theme"
|
||||
html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]
|
||||
|
||||
html_theme_options = {
|
||||
'collapse_navigation': False,
|
||||
}
|
||||
22
docs/index.rst
Normal file
22
docs/index.rst
Normal file
@ -0,0 +1,22 @@
|
||||
.. nvchecker documentation master file, created by
|
||||
sphinx-quickstart on Thu Sep 3 00:19:02 2020.
|
||||
You can adapt this file completely to your liking, but it should at least
|
||||
contain the root `toctree` directive.
|
||||
|
||||
Welcome to nvchecker's documentation!
|
||||
=====================================
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
usage
|
||||
plugin
|
||||
api
|
||||
|
||||
|
||||
Indices and tables
|
||||
==================
|
||||
|
||||
* :ref:`genindex`
|
||||
* :ref:`modindex`
|
||||
* :ref:`search`
|
||||
35
docs/make.bat
Normal file
35
docs/make.bat
Normal file
@ -0,0 +1,35 @@
|
||||
@ECHO OFF
|
||||
|
||||
pushd %~dp0
|
||||
|
||||
REM Command file for Sphinx documentation
|
||||
|
||||
if "%SPHINXBUILD%" == "" (
|
||||
set SPHINXBUILD=sphinx-build
|
||||
)
|
||||
set SOURCEDIR=.
|
||||
set BUILDDIR=_build
|
||||
|
||||
if "%1" == "" goto help
|
||||
|
||||
%SPHINXBUILD% >NUL 2>NUL
|
||||
if errorlevel 9009 (
|
||||
echo.
|
||||
echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
|
||||
echo.installed, then set the SPHINXBUILD environment variable to point
|
||||
echo.to the full path of the 'sphinx-build' executable. Alternatively you
|
||||
echo.may add the Sphinx directory to PATH.
|
||||
echo.
|
||||
echo.If you don't have Sphinx installed, grab it from
|
||||
echo.http://sphinx-doc.org/
|
||||
exit /b 1
|
||||
)
|
||||
|
||||
%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
|
||||
goto end
|
||||
|
||||
:help
|
||||
%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
|
||||
|
||||
:end
|
||||
popd
|
||||
81
docs/plugin.rst
Normal file
81
docs/plugin.rst
Normal file
@ -0,0 +1,81 @@
|
||||
How to develop a source plugin for nvchecker
|
||||
============================================
|
||||
|
||||
.. contents::
|
||||
:local:
|
||||
|
||||
Source plugins enable nvchecker to discover software version strings in
|
||||
additional ways.
|
||||
|
||||
Where to put the plugins
|
||||
------------------------
|
||||
|
||||
They are Python modules put in any directories named ``nvchecker_source`` in
|
||||
``sys.path``. This is called namespace packages introduced by `PEP 420 <https:
|
||||
//www.python.org/dev/peps/pep-0420/>`_. For local use,
|
||||
``~/.local/lib/pythonX.Y/site-packages/nvchecker_source`` is a good place, or
|
||||
you can define the ``PYTHONPATH`` environment variable and put nvchecker source
|
||||
plugins there inside a ``nvchecker_source`` directory.
|
||||
|
||||
Plugins are referenced by their names in the configuration file (``source = "xxx"``).
|
||||
If multiple plugins have the same name, the first one in ``sys.path`` will be used.
|
||||
|
||||
How to write a simple plugin
|
||||
----------------------------
|
||||
|
||||
For simple situations, you need to define an async function with the following signature::
|
||||
|
||||
async def get_version(
|
||||
name: str, conf: Entry, *,
|
||||
cache: AsyncCache, keymanager: KeyManager,
|
||||
**kwargs,
|
||||
) -> VersionResult:
|
||||
...
|
||||
|
||||
Those types are imported from :mod:`nvchecker.api`.
|
||||
|
||||
``name`` is the table keys in the configuration file, and ``conf`` is a dict of
|
||||
the content of that table. You should not modify this dict.
|
||||
|
||||
``cache`` is an :class:`AsyncCache <nvchecker.api.AsyncCache>` object that
|
||||
caches results for you. Every plugin has its own ``cache`` object so that cache
|
||||
keys won't conflict.
|
||||
|
||||
``keymanager`` is a :class:`KeyManager <nvchecker.api.KeyManager>` object that
|
||||
you can call :meth:`.get_key(name) <nvchecker.api.KeyManager.get_key>` to get
|
||||
the key (token) from the keyfile.
|
||||
|
||||
There may be additional keyword arguments in the future so ``**kwargs`` should be used.
|
||||
|
||||
If you want to send an HTTP request, it's preferred to use :meth:
|
||||
`cache.get_json <nvchecker.api.AsyncCache.get_json>` or the :data:
|
||||
`nvchecker.api.session` object. It will use the auto-selected HTTP backend and
|
||||
handle the ``proxy`` option automatically.
|
||||
|
||||
For details about these objects, see :mod:`the API documentation <nvchecker.api>
|
||||
`, or take existing source plugins as examples.
|
||||
|
||||
How to write a more powerful plugin
|
||||
-----------------------------------
|
||||
|
||||
You may want more control in your source plugin, e.g. to do batch requests. To
|
||||
do this, you provide a class instead::
|
||||
|
||||
class Worker(BaseWorker):
|
||||
async def run(self) -> None:
|
||||
...
|
||||
|
||||
|
||||
You will have the following in the attributes::
|
||||
|
||||
token_q: Queue[bool],
|
||||
result_q: Queue[RawResult],
|
||||
tasks: List[Tuple[str, Entry]],
|
||||
keymanager: KeyManager,
|
||||
|
||||
You are expected to process :attr:`tasks <nvchecker.api.BaseWorker.tasks>` and
|
||||
put results in :attr:`result_q <nvchecker.api.BaseWorker.result_q>`. See
|
||||
``nvchecker_source/none.py`` for the simplest example, and
|
||||
``nvchecker_source/aur.py`` for a complete, batching example.
|
||||
|
||||
For details about these objects, see :mod:`the API documentation <nvchecker.api>`.
|
||||
630
docs/usage.rst
Normal file
630
docs/usage.rst
Normal file
@ -0,0 +1,630 @@
|
||||
Usage of nvchecker commands
|
||||
===========================
|
||||
|
||||
**nvchecker** (short for *new version checker*) is for checking if a new version of some software has been released.
|
||||
|
||||
This is the version 2.0 branch that is in beta. Things may still change without backward-compatibility.
|
||||
|
||||
For the old version 1.x, please switch to the ``v1.x`` branch.
|
||||
|
||||
.. image:: https://travis-ci.org/lilydjwg/nvchecker.svg?branch=master
|
||||
:alt: Build Status
|
||||
:target: https://travis-ci.org/lilydjwg/nvchecker
|
||||
.. image:: https://badge.fury.io/py/nvchecker.svg
|
||||
:alt: PyPI version
|
||||
:target: https://badge.fury.io/py/nvchecker
|
||||
|
||||
.. contents::
|
||||
:local:
|
||||
|
||||
Dependency
|
||||
----------
|
||||
- Python 3.7+
|
||||
- Python library: structlog, toml, appdirs
|
||||
- One of these Python library combinations (ordered by preference):
|
||||
|
||||
* tornado + pycurl
|
||||
* aiohttp
|
||||
* httpx with http2 support
|
||||
* tornado
|
||||
|
||||
- All commands used in your software version configuration files
|
||||
|
||||
Install and Run
|
||||
---------------
|
||||
To install::
|
||||
|
||||
pip3 install nvchecker
|
||||
|
||||
To use the latest code, you can also clone this repository and run::
|
||||
|
||||
python3 setup.py install
|
||||
|
||||
To see available options::
|
||||
|
||||
nvchecker --help
|
||||
|
||||
Run with one or more software version files::
|
||||
|
||||
nvchecker -c config_file
|
||||
|
||||
You normally will like to specify some "version record files"; see below.
|
||||
|
||||
JSON logging
|
||||
~~~~~~~~~~~~
|
||||
With ``--logger=json`` or ``--logger=both``, you can get a structured logging
|
||||
for programmatically consuming. You can use ``--json-log-fd=FD`` to specify the
|
||||
file descriptor to send logs to (take care to do line buffering). The logging
|
||||
level option (``-l`` or ``--logging``) doesn't take effect with this.
|
||||
|
||||
The JSON log is one JSON string per line. The following documented events and
|
||||
fields are stable, undocumented ones may change without notice.
|
||||
|
||||
event=updated
|
||||
An update is detected. Fields ``name``, ``old_version`` and ``version`` are
|
||||
available. ``old_version`` maybe ``null``.
|
||||
|
||||
event=up-to-date
|
||||
There is no update. Fields ``name`` and ``version`` are available.
|
||||
|
||||
event=no-result
|
||||
No version is detected. There may be an error. Fields ``name`` is available.
|
||||
|
||||
level=error
|
||||
There is an error. Fields ``name`` and ``exc_info`` may be available to give
|
||||
further information.
|
||||
|
||||
Upgrade from 1.x version
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
There are several backward-incompatible changes from the previous 1.x version.
|
||||
|
||||
1. Version 2.x requires Python 3.7+ to run.
|
||||
2. The command syntax changes a bit. You need to use a ``-c`` switch to specify your software version configuration file (or use the default).
|
||||
3. The configuration file format has been changed from ini to `toml`_. You can use the ``nvchecker-ini2toml`` script to convert your old configuration files. However, comments and formatting will be lost, and some options may not be converted correctly.
|
||||
4. Several options have been renamed. ``max_concurrent`` to ``max_concurrency``, and all option names have their ``-`` be replaced with ``_``.
|
||||
5. All software configuration tables need a ``source`` option to specify which source is to be used rather than being figured out from option names in use. This enables additional source plugins to be discovered.
|
||||
6. The version record files have been changed to use JSON format (the old format will be converted on writing).
|
||||
7. The ``vcs`` source is removed. (It's available inside `lilac <https://github.com/archlinuxcn/lilac>`_ at the moment.)
|
||||
8. ``include_tags_pattern`` and ``ignored_tags`` are removed. Use :ref:`list options` instead.
|
||||
|
||||
Version Record Files
|
||||
--------------------
|
||||
Version record files record which version of the software you know or is available. They are a simple JSON object mapping software names to known versions.
|
||||
|
||||
The ``nvtake`` Command
|
||||
~~~~~~~~~~~~~~~~~~~~~~
|
||||
This command helps to manage version record files. It reads both old and new version record files, and a list of names given on the commandline. It then update the versions of those names in the old version record file.
|
||||
|
||||
This helps when you have known (and processed) some of the updated software, but not all. You can tell nvchecker that via this command instead of editing the file by hand.
|
||||
|
||||
This command will help most if you specify where you version record files are in your config file. See below for how to use a config file.
|
||||
|
||||
The ``nvcmp`` Command
|
||||
~~~~~~~~~~~~~~~~~~~~~
|
||||
This command compares the ``newver`` file with the ``oldver`` one and prints out any differences as updates, e.g.::
|
||||
|
||||
$ nvcmp -c sample_source.toml
|
||||
Sparkle Test App None -> 2.0
|
||||
test 0.0 -> 0.1
|
||||
|
||||
Configuration Files
|
||||
-------------------
|
||||
The software version source files are in `toml`_ format. The *key name* is the name of the software. Following fields are used to tell nvchecker how to determine the current version of that software.
|
||||
|
||||
See ``sample_source.toml`` for an example.
|
||||
|
||||
Configuration Table
|
||||
~~~~~~~~~~~~~~~~~~~
|
||||
A special table named ``__config__`` provides some configuration options.
|
||||
|
||||
Relative path are relative to the source files, and ``~`` and environmental variables are expanded.
|
||||
|
||||
Currently supported options are:
|
||||
|
||||
oldver
|
||||
Specify a version record file containing the old version info.
|
||||
|
||||
newver
|
||||
Specify a version record file to store the new version info.
|
||||
|
||||
proxy
|
||||
The HTTP proxy to use. The format is ``proto://host:port``, e.g. ``http://localhost:8087``. Different backends have different level support for this, e.g. with ``pycurl`` you can use ``socks5h://host:port`` proxies.
|
||||
|
||||
max_concurrency
|
||||
Max number of concurrent jobs. Default: 20.
|
||||
|
||||
keyfile
|
||||
Specify an ini config file containing key (token) information. This file
|
||||
should contain a ``keys`` table, mapping key names to key values. See
|
||||
specific source for the key name(s) to use.
|
||||
|
||||
Global Options
|
||||
~~~~~~~~~~~~~~
|
||||
The following options apply to all check sources.
|
||||
|
||||
prefix
|
||||
Strip the prefix string if the version string starts with it. Otherwise the
|
||||
version string is returned as-is.
|
||||
|
||||
from_pattern, to_pattern
|
||||
Both are Python-compatible regular expressions. If ``from_pattern`` is found
|
||||
in the version string, it will be replaced with ``to_pattern``.
|
||||
|
||||
missing_ok
|
||||
Suppress warnings and errors if a version checking module finds nothing.
|
||||
Currently only ``regex`` supports it.
|
||||
|
||||
proxy
|
||||
The HTTP proxy to use. The format is ``proto://host:port``, e.g.
|
||||
``http://localhost:8087``. Different backends have different level support
|
||||
for this, e.g. with ``pycurl`` you can use ``socks5h://host:port`` proxies.
|
||||
|
||||
Set it to ``""`` (empty string) to override the global setting.
|
||||
|
||||
This only works when the source implementation uses the builtin HTTP client,
|
||||
and doesn't work with the ``aur`` source because it's batched. However the
|
||||
global proxy config applies.
|
||||
|
||||
user_agent
|
||||
The user agent string to use for HTTP requests.
|
||||
|
||||
tries
|
||||
Try specified times when a network error occurs. Default is ``1``.
|
||||
|
||||
This only works when the source implementation uses the builtin HTTP client.
|
||||
|
||||
If both ``prefix`` and ``from_pattern``/``to_pattern`` are used,
|
||||
``from_pattern``/``to_pattern`` are ignored. If you want to strip the prefix
|
||||
and then do something special, just use ``from_pattern```/``to_pattern``. For
|
||||
example, the transformation of ``v1_1_0`` => ``1.1.0`` can be achieved with
|
||||
``from_pattern = v(\d+)_(\d+)_(\d+)`` and ``to_pattern = \1.\2.\3``.
|
||||
|
||||
.. _list options:
|
||||
|
||||
List Options
|
||||
~~~~~~~~~~~~
|
||||
|
||||
The following options apply to sources that return a list. See
|
||||
individual source tables to determine whether they are
|
||||
supported.
|
||||
|
||||
include_regex
|
||||
Only consider version strings that match the given regex. The whole string
|
||||
should match the regex. Be sure to use ``.*`` when you mean it!
|
||||
|
||||
exclude_regex
|
||||
Don't consider version strings that match the given regex. The whole string
|
||||
should match the regex. Be sure to use ``.*`` when you mean it! This option
|
||||
has higher precedence that ``include_regex``; that is, if matched by this
|
||||
one, it's excluded even it's also matched by ``include_regex``.
|
||||
|
||||
sort_version_key
|
||||
Sort the version string using this key function. Choose between
|
||||
``parse_version`` and ``vercmp``. Default value is ``parse_version``.
|
||||
``parse_version`` use ``pkg_resources.parse_version``. ``vercmp`` use
|
||||
``pyalpm.vercmp``.
|
||||
|
||||
ignored
|
||||
Version strings that are explicitly ignored, separated by whitespace. This
|
||||
can be useful to avoid some known mis-named versions, so newer ones won't be
|
||||
"overridden" by the old broken ones.
|
||||
|
||||
Search in a Webpage
|
||||
~~~~~~~~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "regex"
|
||||
|
||||
Search through a specific webpage for the version string. This type of version finding has these fields:
|
||||
|
||||
url
|
||||
The URL of the webpage to fetch.
|
||||
|
||||
encoding
|
||||
(*Optional*) The character encoding of the webpage, if ``latin1`` is not appropriate.
|
||||
|
||||
regex
|
||||
A regular expression used to find the version string.
|
||||
|
||||
It can have zero or one capture group. The capture group or the whole match is the version string.
|
||||
|
||||
When multiple version strings are found, the maximum of those is chosen.
|
||||
|
||||
This source supports :ref:`list options`.
|
||||
|
||||
Find with a Command
|
||||
~~~~~~~~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "cmd"
|
||||
|
||||
Use a shell command line to get the version. The output is striped first, so trailing newlines do not bother.
|
||||
|
||||
cmd
|
||||
The command line to use. This will run with the system's standard shell (i.e. ``/bin/sh``).
|
||||
|
||||
Check AUR
|
||||
~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "aur"
|
||||
|
||||
Check `Arch User Repository <https://aur.archlinux.org/>`_ for updates.
|
||||
Per-item proxy setting doesn't work for this because several items will be
|
||||
batched into one request.
|
||||
|
||||
aur
|
||||
The package name in AUR. If empty, use the name of software (the *table name*).
|
||||
|
||||
strip_release
|
||||
Strip the release part.
|
||||
|
||||
use_last_modified
|
||||
Append last modified time to the version.
|
||||
|
||||
Check GitHub
|
||||
~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "github"
|
||||
|
||||
Check `GitHub <https://github.com/>`_ for updates. The version returned is in
|
||||
date format ``%Y%m%d.%H%M%S``, e.g. ``20130701.012212``, unless ``use_latest_release``
|
||||
or ``use_max_tag`` is used. See below.
|
||||
|
||||
github
|
||||
The github repository, with author, e.g. ``lilydjwg/nvchecker``.
|
||||
|
||||
branch
|
||||
Which branch to track? Default: ``master``.
|
||||
|
||||
path
|
||||
Only commits containing this file path will be returned.
|
||||
|
||||
use_latest_release
|
||||
Set this to ``true`` to check for the latest release on GitHub.
|
||||
|
||||
GitHub releases are not the same with git tags. You'll see big version names
|
||||
and descriptions in the release page for such releases, e.g.
|
||||
`zfsonlinux/zfs's <https://github.com/zfsonlinux/zfs/releases>`_, and those
|
||||
small ones like `nvchecker's <https://github.com/lilydjwg/nvchecker/releases>`_
|
||||
are only git tags that should use ``use_max_tag`` below.
|
||||
|
||||
Will return the release name instead of date.
|
||||
|
||||
use_latest_tag
|
||||
Set this to ``true`` to check for the latest tag on GitHub.
|
||||
|
||||
This requires a token because it's using the v4 GraphQL API.
|
||||
|
||||
query
|
||||
When ``use_latest_tag`` is ``true``, this sets a query for the tag. The exact
|
||||
matching method is not documented by GitHub.
|
||||
|
||||
use_max_tag
|
||||
Set this to ``true`` to check for the max tag on GitHub. Unlike
|
||||
``use_latest_release``, this option includes both annotated tags and
|
||||
lightweight ones, and return the largest one sorted by the
|
||||
``sort_version_key`` option. Will return the tag name instead of date.
|
||||
|
||||
token
|
||||
A personal authorization token used to call the API.
|
||||
|
||||
An authorization token may be needed in order to use ``use_latest_tag`` or to
|
||||
request more frequently than anonymously.
|
||||
|
||||
To set an authorization token, you can set:
|
||||
|
||||
- a key named ``github`` in the keyfile
|
||||
- the token option
|
||||
|
||||
This source supports :ref:`list options` when ``use_max_tag`` is set.
|
||||
|
||||
Check Gitea
|
||||
~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "gitea"
|
||||
|
||||
Check `Gitea <https://gitea.com/>`_ for updates. The version returned is in date format ``%Y%m%d``, e.g. ``20130701``,
|
||||
unless ``use_max_tag`` is used. See below.
|
||||
|
||||
gitea
|
||||
The gitea repository, with author, e.g. ``gitea/tea``.
|
||||
|
||||
branch
|
||||
Which branch to track? Default: ``master``.
|
||||
|
||||
use_max_tag
|
||||
Set this to ``true`` to check for the max tag on Gitea. Will return the biggest one
|
||||
sorted by ``pkg_resources.parse_version``. Will return the tag name instead of date.
|
||||
|
||||
host
|
||||
Hostname for self-hosted Gitea instance.
|
||||
|
||||
token
|
||||
Gitea authorization token used to call the API.
|
||||
|
||||
To set an authorization token, you can set:
|
||||
|
||||
- a key named ``gitea_{host}`` in the keyfile, where ``host`` is all-lowercased host name
|
||||
- the token option
|
||||
|
||||
This source supports :ref:`list options` when ``use_max_tag`` is set.
|
||||
|
||||
Check BitBucket
|
||||
~~~~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "bitbucket"
|
||||
|
||||
Check `BitBucket <https://bitbucket.org/>`_ for updates. The version returned
|
||||
is in date format ``%Y%m%d``, e.g. ``20130701``, unless ``use_max_tag`` is used. See below.
|
||||
|
||||
bitbucket
|
||||
The bitbucket repository, with author, e.g. ``lilydjwg/dotvim``.
|
||||
|
||||
branch
|
||||
Which branch to track? Default is the repository's default.
|
||||
|
||||
use_max_tag
|
||||
Set this to ``true`` to check for the max tag on BitBucket. Will return the biggest one
|
||||
sorted by ``pkg_resources.parse_version``. Will return the tag name instead of date.
|
||||
|
||||
max_page
|
||||
How many pages do we search for the max tag? Default is 3. This works when
|
||||
``use_max_tag`` is set.
|
||||
|
||||
This source supports :ref:`list options` when ``use_max_tag`` is set.
|
||||
|
||||
Check GitLab
|
||||
~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "gitlab"
|
||||
|
||||
Check `GitLab <https://gitlab.com/>`_ for updates. The version returned is in date format ``%Y%m%d``, e.g. ``20130701``,
|
||||
unless ``use_max_tag`` is used. See below.
|
||||
|
||||
gitlab
|
||||
The gitlab repository, with author, e.g. ``Deepin/deepin-music``.
|
||||
|
||||
branch
|
||||
Which branch to track? Default: ``master``.
|
||||
|
||||
use_max_tag
|
||||
Set this to ``true`` to check for the max tag on GitLab. Will return the biggest one
|
||||
sorted by ``pkg_resources.parse_version``. Will return the tag name instead of date.
|
||||
|
||||
host
|
||||
Hostname for self-hosted GitLab instance.
|
||||
|
||||
token
|
||||
GitLab authorization token used to call the API.
|
||||
|
||||
To set an authorization token, you can set:
|
||||
|
||||
- a key named ``gitlab_{host}`` in the keyfile, where ``host`` is all-lowercased host name
|
||||
- the token option
|
||||
|
||||
This source supports :ref:`list options` when ``use_max_tag`` is set.
|
||||
|
||||
Check PyPI
|
||||
~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "pypi"
|
||||
|
||||
Check `PyPI <https://pypi.python.org/>`_ for updates.
|
||||
|
||||
pypi
|
||||
The name used on PyPI, e.g. ``PySide``.
|
||||
|
||||
use_pre_release
|
||||
Whether to accept pre release. Default is false.
|
||||
|
||||
Check RubyGems
|
||||
~~~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "gems"
|
||||
|
||||
Check `RubyGems <https://rubygems.org/>`_ for updates.
|
||||
|
||||
gems
|
||||
The name used on RubyGems, e.g. ``sass``.
|
||||
|
||||
Check NPM Registry
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "npm"
|
||||
|
||||
Check `NPM Registry <https://registry.npmjs.org/>`_ for updates.
|
||||
|
||||
npm
|
||||
The name used on NPM Registry, e.g. ``coffee-script``.
|
||||
|
||||
Check Hackage
|
||||
~~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "hackage"
|
||||
|
||||
Check `Hackage <https://hackage.haskell.org/>`_ for updates.
|
||||
|
||||
hackage
|
||||
The name used on Hackage, e.g. ``pandoc``.
|
||||
|
||||
Check CPAN
|
||||
~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "cpan"
|
||||
|
||||
Check `MetaCPAN <https://metacpan.org/>`_ for updates.
|
||||
|
||||
cpan
|
||||
The name used on CPAN, e.g. ``YAML``.
|
||||
|
||||
Check Packagist
|
||||
~~~~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "packagist"
|
||||
|
||||
Check `Packagist <https://packagist.org/>`_ for updates.
|
||||
|
||||
packagist
|
||||
The name used on Packagist, e.g. ``monolog/monolog``.
|
||||
|
||||
Check Local Pacman Database
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "pacman"
|
||||
|
||||
This is used when you run ``nvchecker`` on an Arch Linux system and the program always keeps up with a package in your configured repositories for `Pacman`_.
|
||||
|
||||
pacman
|
||||
The package name to reference to.
|
||||
|
||||
strip_release
|
||||
Strip the release part.
|
||||
|
||||
Check Arch Linux official packages
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "archpkg"
|
||||
|
||||
This enables you to track the update of `Arch Linux official packages <https://www.archlinux.org/packages/>`_, without needing of pacman and an updated local Pacman databases.
|
||||
|
||||
archpkg
|
||||
Name of the Arch Linux package.
|
||||
|
||||
strip_release
|
||||
Strip the release part, only return part before ``-``.
|
||||
|
||||
provided
|
||||
Instead of the package version, return the version this package provides. Its value is what the package provides, and ``strip-release`` takes effect too. This is best used with libraries.
|
||||
|
||||
Check Debian Linux official packages
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "debianpkg"
|
||||
|
||||
This enables you to track the update of `Debian Linux official packages <https://packages.debian.org>`_, without needing of apt and an updated local APT database.
|
||||
|
||||
debianpkg
|
||||
Name of the Debian Linux source package.
|
||||
|
||||
suite
|
||||
Name of the Debian release (jessie, wheezy, etc, defaults to sid)
|
||||
|
||||
strip_release
|
||||
Strip the release part.
|
||||
|
||||
Check Ubuntu Linux official packages
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "ubuntupkg"
|
||||
|
||||
This enables you to track the update of `Ubuntu Linux official packages <https://packages.ubuntu.com/>`_, without needing of apt and an updated local APT database.
|
||||
|
||||
ubuntupkg
|
||||
Name of the Ubuntu Linux source package.
|
||||
|
||||
suite
|
||||
Name of the Ubuntu release (xenial, zesty, etc, defaults to None, which means no limit on suite)
|
||||
|
||||
strip_release
|
||||
Strip the release part.
|
||||
|
||||
Check Repology
|
||||
~~~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "repology"
|
||||
|
||||
This enables you to track updates from `Repology <https://repology.org/>`_ (repology.org).
|
||||
|
||||
repology
|
||||
Name of the ``project`` to check.
|
||||
|
||||
repo
|
||||
Check the version in this repo. This field is required.
|
||||
|
||||
Check Anitya
|
||||
~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "anitya"
|
||||
|
||||
This enables you to track updates from `Anitya <https://release-monitoring.org/>`_ (release-monitoring.org).
|
||||
|
||||
anitya
|
||||
``distro/package``, where ``distro`` can be a lot of things like "fedora", "arch linux", "gentoo", etc. ``package`` is the package name of the chosen distribution.
|
||||
|
||||
Check Android SDK
|
||||
~~~~~~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "android_sdk"
|
||||
|
||||
This enables you to track updates of Android SDK packages listed in ``sdkmanager --list``.
|
||||
|
||||
android_sdk
|
||||
The package path prefix. This value is matched against the ``path`` attribute in all <remotePackage> nodes in an SDK manifest XML. The first match is used for version comparisons.
|
||||
|
||||
repo
|
||||
Should be one of ``addon`` or ``package``. Packages in ``addon2-1.xml`` use ``addon`` and packages in ``repository2-1.xml`` use ``package``.
|
||||
|
||||
Check Sparkle framework
|
||||
~~~~~~~~~~~~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "sparkle"
|
||||
|
||||
This enables you to track updates of macOS applications which using `Sparkle framework <https://sparkle-project.org/>`_.
|
||||
|
||||
sparkle
|
||||
The url of the sparkle appcast.
|
||||
|
||||
Check Pagure
|
||||
~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "pagure"
|
||||
|
||||
This enables you to check updates from `Pagure <https://pagure.io>`_.
|
||||
|
||||
pagure
|
||||
The project name, optionally with a namespace.
|
||||
|
||||
host
|
||||
Hostname of alternative instance like src.fedoraproject.org.
|
||||
|
||||
This source returns tags and supports :ref:`list options`.
|
||||
|
||||
Manually updating
|
||||
~~~~~~~~~~~~~~~~~
|
||||
::
|
||||
|
||||
source = "manual"
|
||||
|
||||
This enables you to manually specify the version (maybe because you want to approve each release before it gets to the script).
|
||||
|
||||
manual
|
||||
The version string.
|
||||
|
||||
Extending
|
||||
~~~~~~~~~
|
||||
It's possible to extend the supported sources by writing
|
||||
plugins. See :doc:`plugin` for documentation.
|
||||
|
||||
.. _Pacman: https://wiki.archlinux.org/index.php/Pacman
|
||||
.. _toml: https://toml.io/
|
||||
@ -2,10 +2,9 @@
|
||||
# Copyright (c) 2020 lilydjwg <lilydjwg@gmail.com>, et al.
|
||||
|
||||
from .httpclient import session, TemporaryError # type: ignore
|
||||
from .core import GetVersionError
|
||||
from .util import (
|
||||
Entry, BaseWorker, RawResult, VersionResult,
|
||||
AsyncCache, KeyManager,
|
||||
AsyncCache, KeyManager, GetVersionError,
|
||||
)
|
||||
from .sortversion import sort_version_keys
|
||||
from .ctxvars import tries, proxy
|
||||
from .ctxvars import tries, proxy, user_agent
|
||||
|
||||
@ -10,34 +10,52 @@ from ..ctxvars import tries, proxy, user_agent
|
||||
logger = structlog.get_logger(logger_name=__name__)
|
||||
|
||||
class Response:
|
||||
'''The response of an HTTP request.
|
||||
|
||||
.. py:attribute:: body
|
||||
:type: bytes
|
||||
'''
|
||||
def __init__(self, body):
|
||||
self.body = body
|
||||
|
||||
def json(self):
|
||||
'''Convert reponse content to JSON.'''
|
||||
return _json.loads(self.body.decode('utf-8'))
|
||||
|
||||
class BaseSession:
|
||||
'''The base class for different HTTP backend.'''
|
||||
async def get(self, *args, **kwargs):
|
||||
'''Shortcut for ``GET`` request.'''
|
||||
return await self.request(
|
||||
method='GET', *args, **kwargs)
|
||||
|
||||
async def post(self, *args, **kwargs):
|
||||
'''Shortcut for ``POST`` request.'''
|
||||
return await self.request(
|
||||
method='POST', *args, **kwargs)
|
||||
|
||||
async def request(self, *args, **kwargs):
|
||||
async def request(
|
||||
self, url: str, *,
|
||||
method: str,
|
||||
headers: Dict[str, str] = {},
|
||||
params = (),
|
||||
json = None,
|
||||
) -> Response:
|
||||
t = tries.get()
|
||||
p = proxy.get()
|
||||
ua = user_agent.get()
|
||||
|
||||
headers = kwargs.setdefault('headers', {})
|
||||
headers['User-Agent'] = ua
|
||||
headers.setdefault('User-Agent', ua)
|
||||
|
||||
for i in range(1, t+1):
|
||||
try:
|
||||
return await self.request_impl(
|
||||
url,
|
||||
method = method,
|
||||
headers = headers,
|
||||
params = params,
|
||||
json = json,
|
||||
proxy = p or None,
|
||||
*args, **kwargs,
|
||||
)
|
||||
except TemporaryError as e:
|
||||
if i == t:
|
||||
@ -47,6 +65,8 @@ class BaseSession:
|
||||
tries = i, exc_info = e)
|
||||
continue
|
||||
|
||||
raise Exception('shoud not reach')
|
||||
|
||||
async def request_impl(
|
||||
self, url: str, *,
|
||||
method: str,
|
||||
@ -55,9 +75,11 @@ class BaseSession:
|
||||
params = (),
|
||||
json = None,
|
||||
) -> Response:
|
||||
''':meta private:'''
|
||||
raise NotImplementedError
|
||||
|
||||
class TemporaryError(Exception):
|
||||
'''A temporary error (e.g. network error) happens.'''
|
||||
def __init__(self, code, message, response):
|
||||
self.code = code
|
||||
self.message = message
|
||||
|
||||
@ -13,6 +13,7 @@ from typing import (
|
||||
)
|
||||
from pathlib import Path
|
||||
import contextvars
|
||||
import abc
|
||||
|
||||
import toml
|
||||
import structlog
|
||||
@ -25,11 +26,20 @@ from .ctxvars import user_agent as ctx_ua
|
||||
logger = structlog.get_logger(logger_name=__name__)
|
||||
|
||||
Entry = Dict[str, Any]
|
||||
Entry.__doc__ = '''The configuration `dict` for an entry.'''
|
||||
Entries = Dict[str, Entry]
|
||||
VersData = Dict[str, str]
|
||||
VersionResult = Union[None, str, List[str], Exception]
|
||||
VersionResult.__doc__ = '''The result of a `get_version` check.
|
||||
|
||||
* `None` - No version found.
|
||||
* `str` - A single version string is found.
|
||||
* `List[str]` - Multiple version strings are found. :ref:`list options` will be applied.
|
||||
* `Exception` - An error occurred.
|
||||
'''
|
||||
|
||||
class KeyManager:
|
||||
'''Manages data in the keyfile.'''
|
||||
def __init__(
|
||||
self, file: Optional[Path],
|
||||
) -> None:
|
||||
@ -41,19 +51,49 @@ class KeyManager:
|
||||
self.keys = keys
|
||||
|
||||
def get_key(self, name: str) -> Optional[str]:
|
||||
'''Get the named key (token) in the keyfile.'''
|
||||
return self.keys.get(name)
|
||||
|
||||
class RawResult(NamedTuple):
|
||||
'''The unprocessed result from a check.'''
|
||||
name: str
|
||||
version: VersionResult
|
||||
conf: Entry
|
||||
|
||||
RawResult.name.__doc__ = 'The name (table name) of the entry.'
|
||||
RawResult.version.__doc__ = 'The result from the check.'
|
||||
RawResult.conf.__doc__ = 'The entry configuration (table content) of the entry.'
|
||||
|
||||
class Result(NamedTuple):
|
||||
name: str
|
||||
version: str
|
||||
conf: Entry
|
||||
|
||||
class BaseWorker:
|
||||
'''The base class for defining `Worker` classes for source plugins.
|
||||
|
||||
.. py:attribute:: token_q
|
||||
:type: Queue[bool]
|
||||
|
||||
This is the rate-limiting queue. Workers should obtain one token before doing one unit of work.
|
||||
|
||||
.. py:attribute:: result_q
|
||||
:type: Queue[RawResult]
|
||||
|
||||
Results should be put into this queue.
|
||||
|
||||
.. py:attribute:: tasks
|
||||
:type: List[Tuple[str, Entry]]
|
||||
|
||||
A list of tasks for the `Worker` to complete. Every task consists of
|
||||
a tuple for the task name (table name in the configuration file) and the
|
||||
content of that table (as a `dict`).
|
||||
|
||||
.. py:attribute:: keymanager
|
||||
:type: KeyManager
|
||||
|
||||
The `KeyManager` for retrieving keys from the keyfile.
|
||||
'''
|
||||
def __init__(
|
||||
self,
|
||||
token_q: Queue[bool],
|
||||
@ -68,6 +108,8 @@ class BaseWorker:
|
||||
|
||||
@contextlib.asynccontextmanager
|
||||
async def acquire_token(self) -> AsyncGenerator[None, None]:
|
||||
'''A context manager to obtain a token from the `token_q` on entrance and
|
||||
release it on exit.'''
|
||||
token = await self.token_q.get()
|
||||
logger.debug('got token')
|
||||
try:
|
||||
@ -76,8 +118,13 @@ class BaseWorker:
|
||||
await self.token_q.put(token)
|
||||
logger.debug('return token')
|
||||
|
||||
@abc.abstractmethod
|
||||
async def run(self) -> None:
|
||||
'''Run the `tasks`. Subclasses should implement this method.'''
|
||||
raise NotImplementedError
|
||||
|
||||
class AsyncCache:
|
||||
'''A cache for use with async funtions.'''
|
||||
cache: Dict[Hashable, Any]
|
||||
lock: asyncio.Lock
|
||||
|
||||
@ -96,6 +143,10 @@ class AsyncCache:
|
||||
self, url: str, *,
|
||||
headers: Dict[str, str] = {},
|
||||
) -> Any:
|
||||
'''Get specified ``url`` and return the response content as JSON.
|
||||
|
||||
The returned data will be cached for reuse.
|
||||
'''
|
||||
key = '_jsonurl', url, tuple(sorted(headers.items()))
|
||||
return await self.get(
|
||||
key , self._get_json) # type: ignore
|
||||
@ -105,6 +156,13 @@ class AsyncCache:
|
||||
key: Hashable,
|
||||
func: Callable[[Hashable], Coroutine[Any, Any, Any]],
|
||||
) -> Any:
|
||||
'''Run async ``func`` and cache its return value by ``key``.
|
||||
|
||||
The ``key`` should be hashable, and the function will be called with it as
|
||||
its sole argument. For multiple simultaneous calls with the same key, only
|
||||
one will actually be called, and others will wait and return the same
|
||||
(cached) value.
|
||||
'''
|
||||
async with self.lock:
|
||||
cached = self.cache.get(key)
|
||||
if cached is None:
|
||||
@ -180,6 +238,13 @@ class FunctionWorker(BaseWorker):
|
||||
await self.result_q.put(RawResult(name, e, entry))
|
||||
|
||||
class GetVersionError(Exception):
|
||||
'''An error occurred while getting version information.
|
||||
|
||||
Raise this when a known bad situation happens.
|
||||
|
||||
:param msg: The error message.
|
||||
:param kwargs: Arbitrary additional context for the error.
|
||||
'''
|
||||
def __init__(self, msg: str, **kwargs: Any) -> None:
|
||||
self.msg = msg
|
||||
self.kwargs = kwargs
|
||||
|
||||
Loading…
Reference in New Issue
Block a user