From a7fed6c2136abd68fa17325c74579b880049f997 Mon Sep 17 00:00:00 2001 From: Alfredo Deza Date: Mon, 14 Aug 2017 09:55:52 -0400 Subject: [PATCH 1/8] doc: update repo-access to point to ceph-ci.git Signed-off-by: Alfredo Deza --- doc/dev/repo-access.rst | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/doc/dev/repo-access.rst b/doc/dev/repo-access.rst index 0196b0af9e8..e3edf468d78 100644 --- a/doc/dev/repo-access.rst +++ b/doc/dev/repo-access.rst @@ -5,23 +5,21 @@ Special branches ---------------- * ``master``: current tip (integration branch) -* ``next``: pending release (feature frozen, bugfixes only) -* ``last``: last/previous release -* ``dumpling``, ``cuttlefish``, ``bobtail``, ``argonaut``, etc.: stable release branches -* ``dumpling-next``: backports for stable release, pending testing +* Release branches (for example ``luminous``) corresponding to the releases + listed at :doc:`releases <../releases>` Rules ----- The source repos are all on github. -* Any branch pushed to ceph.git will kick off builds that will either - run unit tests or generate packages for gitbuilder.ceph.com. Try +* Any branch pushed to ceph-ci.git will kick off builds that will + generate packages and repositories on shaman.ceph.com. Try not to generate unnecessary load. For private, unreviewed work, only push to branches named ``wip-*``. This avoids colliding with any special branches. -* Nothing should every reach a special branch unless it has been +* Nothing should reach a special branch unless it has been reviewed. * Preferred means of review is via github pull requests to capture any From 44c9698660246b0887c1c023c95aabbe494d5e8b Mon Sep 17 00:00:00 2001 From: Alfredo Deza Date: Mon, 14 Aug 2017 10:06:54 -0400 Subject: [PATCH 2/8] doc/install remove Apache and FastCGI instructions for RGW Signed-off-by: Alfredo Deza --- doc/install/get-packages.rst | 80 ------------------------------------ 1 file changed, 80 deletions(-) diff --git a/doc/install/get-packages.rst b/doc/install/get-packages.rst index 02a24cd14c5..814105b70f2 100644 --- a/doc/install/get-packages.rst +++ b/doc/install/get-packages.rst @@ -43,14 +43,6 @@ also add keys and recommended packages. development, you may get Ceph development packages. See `Add Ceph Development`_ for details. -- **Apache/FastCGI: (Optional)** If you are deploying a - :term:`Ceph Object Storage` service, you must install Apache and FastCGI. - Ceph provides Apache and FastCGI builds that are identical to those available - from Apache, but with 100-continue support. If you want to enable - :term:`Ceph Object Gateway` daemons with 100-continue support, you must - retrieve Apache/FastCGI packages from the Ceph repository. - See `Add Apache/FastCGI`_ for details. - If you intend to download packages manually, see Section `Download Packages`_. @@ -286,78 +278,6 @@ You may view http://gitbuilder.ceph.com directory to see which distributions Ceph supports. -Add Apache/FastCGI -================== - -Ceph Object Gateway works with ordinary Apache and FastCGI libraries. However, -Ceph builds Apache and FastCGI packages that support 100-continue. To use the -Ceph Apache and FastCGI packages, add them to your repository. - - -Debian Packages ---------------- - -Add our Apache and FastCGI packages to your system's list of APT sources if you intend to -use 100-continue. :: - - echo deb http://gitbuilder.ceph.com/apache2-deb-$(lsb_release -sc)-x86_64-basic/ref/master $(lsb_release -sc) main | sudo tee /etc/apt/sources.list.d/ceph-apache.list - echo deb http://gitbuilder.ceph.com/libapache-mod-fastcgi-deb-$(lsb_release -sc)-x86_64-basic/ref/master $(lsb_release -sc) main | sudo tee /etc/apt/sources.list.d/ceph-fastcgi.list - - -RPM Packages ------------- - -You may add a Ceph entry to the ``/etc/yum.repos.d`` directory. Create a -``ceph-apache.repo`` file. In the example below, replace ``{distro}`` with your -Linux distribution (e.g., ``el7``). You may view http://gitbuilder.ceph.com -directory to see which distributions Ceph supports. -:: - - - [apache2-ceph-noarch] - name=Apache noarch packages for Ceph - baseurl=http://gitbuilder.ceph.com/apache2-rpm-{distro}-x86_64-basic/ref/master - enabled=1 - priority=2 - gpgcheck=1 - gpgkey=https://download.ceph.com/keys/autobuild.asc - - [apache2-ceph-source] - name=Apache source packages for Ceph - baseurl=http://gitbuilder.ceph.com/apache2-rpm-{distro}-x86_64-basic/ref/master - enabled=0 - priority=2 - gpgcheck=1 - gpgkey=https://download.ceph.com/keys/autobuild.asc - - -Repeat the forgoing process by creating a ``ceph-fastcgi.repo`` file. :: - - [fastcgi-ceph-basearch] - name=FastCGI basearch packages for Ceph - baseurl=http://gitbuilder.ceph.com/mod_fastcgi-rpm-{distro}-x86_64-basic/ref/master - enabled=1 - priority=2 - gpgcheck=1 - gpgkey=https://download.ceph.com/keys/autobuild.asc - - [fastcgi-ceph-noarch] - name=FastCGI noarch packages for Ceph - baseurl=http://gitbuilder.ceph.com/mod_fastcgi-rpm-{distro}-x86_64-basic/ref/master - enabled=1 - priority=2 - gpgcheck=1 - gpgkey=https://download.ceph.com/keys/autobuild.asc - - [fastcgi-ceph-source] - name=FastCGI source packages for Ceph - baseurl=http://gitbuilder.ceph.com/mod_fastcgi-rpm-{distro}-x86_64-basic/ref/master - enabled=0 - priority=2 - gpgcheck=1 - gpgkey=https://download.ceph.com/keys/autobuild.asc - - Download Packages ================= From 765ab51b13e535cbe3190e46edcabaaf68a325ea Mon Sep 17 00:00:00 2001 From: Alfredo Deza Date: Mon, 14 Aug 2017 10:09:13 -0400 Subject: [PATCH 3/8] doc/install remove mention of autobuild key for dev packages Dev packages are no longer signed, and the key is not used at all Signed-off-by: Alfredo Deza --- doc/install/get-packages.rst | 30 ++++++------------------------ 1 file changed, 6 insertions(+), 24 deletions(-) diff --git a/doc/install/get-packages.rst b/doc/install/get-packages.rst index 814105b70f2..a7746ff27e6 100644 --- a/doc/install/get-packages.rst +++ b/doc/install/get-packages.rst @@ -30,9 +30,7 @@ also add keys and recommended packages. - **Keys: (Recommended)** Whether you add repositories or download packages manually, you should download keys to verify the packages. If you do not get - the keys, you may encounter security warnings. There are two keys: one for - releases (common) and one for development (programmers and QA only). Choose - the key that suits your needs. See `Add Keys`_ for details. + the keys, you may encounter security warnings. See `Add Keys`_ for details. - **Ceph: (Required)** All Ceph deployments require Ceph release packages, except for deployments that use development packages (development, QA, and @@ -52,9 +50,7 @@ Add Keys Add a key to your system's list of trusted keys to avoid a security warning. For major releases (e.g., ``hammer``, ``jewel``) and development releases -(``release-name-rc1``, ``release-name-rc2``), use the ``release.asc`` key. For -development testing packages, use the ``autobuild.asc`` key (developers and -QA). +(``release-name-rc1``, ``release-name-rc2``), use the ``release.asc`` key. APT @@ -65,12 +61,6 @@ To install the ``release.asc`` key, execute the following:: wget -q -O- 'https://download.ceph.com/keys/release.asc' | sudo apt-key add - -To install the ``autobuild.asc`` key, execute the following -(QA and developers only):: - - wget -q -O- 'https://download.ceph.com/keys/autobuild.asc' | sudo apt-key add - - - RPM --- @@ -78,12 +68,6 @@ To install the ``release.asc`` key, execute the following:: sudo rpm --import 'https://download.ceph.com/keys/release.asc' -To install the ``autobuild.asc`` key, execute the following -(QA and developers only):: - - sudo rpm --import 'https://download.ceph.com/keys/autobuild.asc' - - Add Ceph ======== @@ -106,9 +90,8 @@ bugfixes are backported to LTS releases until their retirement. Since retired releases are no longer maintained, we recommend that users upgrade their clusters regularly - preferably to the latest LTS release. -The most recent LTS release is Jewel (10.2.x). - -.. tip:: For international users: There might be a mirror close to you where download Ceph from. For more information see: `Ceph Mirrors`_. +.. tip:: For international users: There might be a mirror close to you where + download Ceph from. For more information see: `Ceph Mirrors`_. Debian Packages --------------- @@ -126,6 +109,7 @@ For early Linux distributions, you may execute the following command:: For earlier Ceph releases, replace ``{release-name}`` with the name with the name of the Ceph release. You may call ``lsb_release -sc`` on the command line to get the short codename, and replace ``{codename}`` in the following command. + :: sudo apt-add-repository 'deb https://download.ceph.com/debian-{release-name}/ {codename} main' @@ -237,7 +221,6 @@ You can download the RPMs directly from:: Add Ceph Development ==================== -Development repositories use the ``autobuild.asc`` key to verify packages. If you are developing Ceph and need to deploy and test specific Ceph branches, ensure that you remove repository entries for major releases first. @@ -270,8 +253,7 @@ below, replace ``{distro}`` with your Linux distribution (e.g., ``el7``), and name=Ceph source packages baseurl=http://gitbuilder.ceph.com/ceph-rpm-{distro}-x86_64-basic/ref/{branch}/SRPMS enabled=0 - gpgcheck=1 - gpgkey=https://download.ceph.com/keys/autobuild.asc + gpgcheck=0 You may view http://gitbuilder.ceph.com directory to see which distributions From 072bdbdf25f48ebe90c9397d1b230ec48f87ee9d Mon Sep 17 00:00:00 2001 From: Alfredo Deza Date: Mon, 14 Aug 2017 10:15:14 -0400 Subject: [PATCH 4/8] doc/install remove repetitive dev release information for rpm distros Signed-off-by: Alfredo Deza --- doc/install/get-packages.rst | 28 ---------------------------- 1 file changed, 28 deletions(-) diff --git a/doc/install/get-packages.rst b/doc/install/get-packages.rst index a7746ff27e6..ebce755f4d5 100644 --- a/doc/install/get-packages.rst +++ b/doc/install/get-packages.rst @@ -171,34 +171,6 @@ over standard packages, so you must ensure that you set gpgkey=https://download.ceph.com/keys/release.asc -For development release packages, you may specify the repository -for development releases instead. :: - - [ceph] - name=Ceph packages for $basearch/$releasever - baseurl=https://download.ceph.com/rpm-testing/{distro}/$basearch - enabled=1 - priority=2 - gpgcheck=1 - gpgkey=https://download.ceph.com/keys/release.asc - - [ceph-noarch] - name=Ceph noarch packages - baseurl=https://download.ceph.com/rpm-testing/{distro}/noarch - enabled=1 - priority=2 - gpgcheck=1 - gpgkey=https://download.ceph.com/keys/release.asc - - [ceph-source] - name=Ceph source packages - baseurl=https://download.ceph.com/rpm-testing/{distro}/SRPMS - enabled=0 - priority=2 - gpgcheck=1 - gpgkey=https://download.ceph.com/keys/release.asc - - For specific packages, you may retrieve them by specifically downloading the release package by name. Our development process generates a new release of Ceph every 3-4 weeks. These packages are faster-moving than the major releases. From 2ef1eb9a00fbb0a5822df9ae29e28bc438156cb7 Mon Sep 17 00:00:00 2001 From: Alfredo Deza Date: Mon, 14 Aug 2017 10:19:57 -0400 Subject: [PATCH 5/8] doc/install update international for non-US Use of "international" is ambiguous, for non-US users, international could mean US. Signed-off-by: Alfredo Deza --- doc/install/get-packages.rst | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/doc/install/get-packages.rst b/doc/install/get-packages.rst index ebce755f4d5..47f8bc18e29 100644 --- a/doc/install/get-packages.rst +++ b/doc/install/get-packages.rst @@ -90,8 +90,8 @@ bugfixes are backported to LTS releases until their retirement. Since retired releases are no longer maintained, we recommend that users upgrade their clusters regularly - preferably to the latest LTS release. -.. tip:: For international users: There might be a mirror close to you where - download Ceph from. For more information see: `Ceph Mirrors`_. +.. tip:: For non-US users: There might be a mirror close to you where + to download Ceph from. For more information see: `Ceph Mirrors`_. Debian Packages --------------- @@ -132,7 +132,9 @@ of Debian and Ubuntu releases supported. :: echo deb https://download.ceph.com/debian-testing/ $(lsb_release -sc) main | sudo tee /etc/apt/sources.list.d/ceph.list -.. tip:: For international users: There might be a mirror close to you where download Ceph from. For more information see: `Ceph Mirrors`_. +.. tip:: For non-US users: There might be a mirror close to you where + to download Ceph from. For more information see: `Ceph Mirrors`_. + RPM Packages ------------ @@ -187,7 +189,8 @@ You can download the RPMs directly from:: https://download.ceph.com/rpm-testing -.. tip:: For international users: There might be a mirror close to you where download Ceph from. For more information see: `Ceph Mirrors`_. +.. tip:: For non-US users: There might be a mirror close to you where + to download Ceph from. For more information see: `Ceph Mirrors`_. Add Ceph Development From 253804b3d6d2995e694c5a3186d5ac3cecfc2160 Mon Sep 17 00:00:00 2001 From: Alfredo Deza Date: Mon, 14 Aug 2017 12:27:21 -0400 Subject: [PATCH 6/8] doc/install describe shaman urls and how to retrieve list and repo files Signed-off-by: Alfredo Deza --- doc/install/get-packages.rst | 65 ++++++++++++++++++++---------------- 1 file changed, 37 insertions(+), 28 deletions(-) diff --git a/doc/install/get-packages.rst b/doc/install/get-packages.rst index 47f8bc18e29..68678d86eb7 100644 --- a/doc/install/get-packages.rst +++ b/doc/install/get-packages.rst @@ -173,11 +173,11 @@ over standard packages, so you must ensure that you set gpgkey=https://download.ceph.com/keys/release.asc -For specific packages, you may retrieve them by specifically downloading the -release package by name. Our development process generates a new release of Ceph -every 3-4 weeks. These packages are faster-moving than the major releases. -Development packages have new features integrated quickly, while still -undergoing several weeks of QA prior to release. +For specific packages, you may retrieve them by downloading the release package +by name. Our development process generates a new release of Ceph every 3-4 +weeks. These packages are faster-moving than the major releases. Development +packages have new features integrated quickly, while still undergoing several +weeks of QA prior to release. The repository package installs the repository details on your local system for use with ``yum``. Replace ``{distro}`` with your Linux distribution, and @@ -200,40 +200,50 @@ If you are developing Ceph and need to deploy and test specific Ceph branches, ensure that you remove repository entries for major releases first. -Debian Packages ---------------- +DEB Packages +------------ -We automatically build Debian and Ubuntu packages for current -development branches in the Ceph source code repository. These -packages are intended for developers and QA only. +We automatically build Ubuntu packages for current development branches in the +Ceph source code repository. These packages are intended for developers and QA +only. -Add our package repository to your system's list of APT sources, but -replace ``{BRANCH}`` with the branch you'd like to use (e.g., chef-3, -wip-hack, master). See `the gitbuilder page`_ for a complete +Add the package repository to your system's list of APT sources, but +replace ``{BRANCH}`` with the branch you'd like to use (e.g., +wip-hack, master). See `the shaman page`_ for a complete list of distributions we build. :: - echo deb http://gitbuilder.ceph.com/ceph-deb-$(lsb_release -sc)-x86_64-basic/ref/{BRANCH} $(lsb_release -sc) main | sudo tee /etc/apt/sources.list.d/ceph.list + curl -L https://shaman.ceph.com/api/repos/ceph/{BRANCH}/latest/ubuntu/$(lsb_release -sc)/repo/ | sudo tee /etc/apt/sources.list.d/shaman.list +.. note:: If the repository is not ready an HTTP 504 will be returned + +The use of ``latest`` in the url, means it will figure out which is the last +commit that has been built. Alternatively, a specific sha1 can be specified. +For Ubuntu Xenial and the master branch of Ceph, it would look like:: + + curl -L https://shaman.ceph.com/api/repos/ceph/master/53e772a45fdf2d211c0c383106a66e1feedec8fd/ubuntu/xenial/repo/ | sudo tee /etc/apt/sources.list.d/shaman.list + + +.. warning:: Development repositories are no longer available after two weeks. RPM Packages ------------ For current development branches, you may add a Ceph entry to the -``/etc/yum.repos.d`` directory. Create a ``ceph.repo`` file. In the example -below, replace ``{distro}`` with your Linux distribution (e.g., ``el7``), and -``{branch}`` with the name of the branch you want to install. :: +``/etc/yum.repos.d`` directory. The `the shaman page`_ can be used to retrieve the full details +of a repo file. It can be retrieved via an HTTP request, for example:: + + curl -L https://shaman.ceph.com/api/repos/ceph/{BRANCH}/latest/centos/7/repo/ | sudo tee /etc/yum.repos.d/shaman.repo + +The use of ``latest`` in the url, means it will figure out which is the last +commit that has been built. Alternatively, a specific sha1 can be specified. +For CentOS 7 and the master branch of Ceph, it would look like:: + + curl -L https://shaman.ceph.com/api/repos/ceph/master/53e772a45fdf2d211c0c383106a66e1feedec8fd/centos/7/repo/ | sudo tee /etc/apt/sources.list.d/shaman.list - [ceph-source] - name=Ceph source packages - baseurl=http://gitbuilder.ceph.com/ceph-rpm-{distro}-x86_64-basic/ref/{branch}/SRPMS - enabled=0 - gpgcheck=0 - - -You may view http://gitbuilder.ceph.com directory to see which distributions -Ceph supports. +.. warning:: Development repositories are no longer available after two weeks. +.. note:: If the repository is not ready an HTTP 504 will be returned Download Packages ================= @@ -306,8 +316,7 @@ line to get the short codename. :: - .. _Install Ceph Object Storage: ../install-storage-cluster .. _the testing Debian repository: https://download.ceph.com/debian-testing/dists -.. _the gitbuilder page: http://gitbuilder.ceph.com +.. _the shaman page: https://shaman.ceph.com .. _Ceph Mirrors: ../mirrors From e7a3bb94fb997c8aab6967b4c3b550629e9803f2 Mon Sep 17 00:00:00 2001 From: Alfredo Deza Date: Wed, 16 Aug 2017 11:06:49 -0400 Subject: [PATCH 7/8] doc/rados/api comment out '...' on ini blocks Signed-off-by: Alfredo Deza --- doc/rados/api/python.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/rados/api/python.rst b/doc/rados/api/python.rst index d7555686b0b..6af81632709 100644 --- a/doc/rados/api/python.rst +++ b/doc/rados/api/python.rst @@ -114,7 +114,7 @@ configuration file example uses the ``client.admin`` keyring you generated with :linenos: [global] - ... + # ... elided configuration keyring=/path/to/keyring/ceph.client.admin.keyring From b2c861ed6245c14e6f9c9ff2b63627a7cb52a21e Mon Sep 17 00:00:00 2001 From: Alfredo Deza Date: Wed, 16 Aug 2017 11:08:59 -0400 Subject: [PATCH 8/8] doc/rados/configuration comment out '...' on ini blocks Signed-off-by: Alfredo Deza --- doc/rados/configuration/network-config-ref.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/rados/configuration/network-config-ref.rst b/doc/rados/configuration/network-config-ref.rst index 2d7f9d6da27..d9b930622bb 100644 --- a/doc/rados/configuration/network-config-ref.rst +++ b/doc/rados/configuration/network-config-ref.rst @@ -198,7 +198,7 @@ section of your Ceph configuration file. .. code-block:: ini [global] - ... + # ... elided configuration public network = {public-network/netmask} @@ -213,7 +213,7 @@ following option to the ``[global]`` section of your Ceph configuration file. .. code-block:: ini [global] - ... + # ... elided configuration cluster network = {cluster-network/netmask} We prefer that the cluster network is **NOT** reachable from the public network