2020-01-06 07:40:50 +00:00
|
|
|
Essentials (tl;dr)
|
|
|
|
==================
|
|
|
|
|
|
|
|
This chapter presents essential information that every Ceph developer needs
|
|
|
|
to know.
|
|
|
|
|
|
|
|
Leads
|
|
|
|
-----
|
|
|
|
|
|
|
|
The Ceph project is led by Sage Weil. In addition, each major project
|
|
|
|
component has its own lead. The following table shows all the leads and
|
|
|
|
their nicks on `GitHub`_:
|
|
|
|
|
|
|
|
.. _github: https://github.com/
|
|
|
|
|
|
|
|
========= ================ =============
|
|
|
|
Scope Lead GitHub nick
|
|
|
|
========= ================ =============
|
|
|
|
Ceph Sage Weil liewegas
|
|
|
|
RADOS Neha Ojha neha-ojha
|
|
|
|
RGW Yehuda Sadeh yehudasa
|
|
|
|
RGW Matt Benjamin mattbenjamin
|
2021-09-16 18:45:35 +00:00
|
|
|
RBD Ilya Dryomov dis
|
2020-01-06 07:40:50 +00:00
|
|
|
CephFS Patrick Donnelly batrick
|
2021-01-27 15:43:59 +00:00
|
|
|
Dashboard Ernesto Puerta epuertat
|
2020-01-06 07:40:50 +00:00
|
|
|
MON Joao Luis jecluis
|
|
|
|
Build/Ops Ken Dreyer ktdreyer
|
2020-07-15 10:59:06 +00:00
|
|
|
Docs Zac Dover zdover23
|
2020-01-06 07:40:50 +00:00
|
|
|
========= ================ =============
|
|
|
|
|
|
|
|
The Ceph-specific acronyms in the table are explained in
|
|
|
|
:doc:`/architecture`.
|
|
|
|
|
|
|
|
History
|
|
|
|
-------
|
|
|
|
|
|
|
|
See the `History chapter of the Wikipedia article`_.
|
|
|
|
|
|
|
|
.. _`History chapter of the Wikipedia article`: https://en.wikipedia.org/wiki/Ceph_%28software%29#History
|
|
|
|
|
|
|
|
Licensing
|
|
|
|
---------
|
|
|
|
|
|
|
|
Ceph is free software.
|
|
|
|
|
|
|
|
Unless stated otherwise, the Ceph source code is distributed under the
|
|
|
|
terms of the LGPL2.1 or LGPL3.0. For full details, see the file
|
|
|
|
`COPYING`_ in the top-level directory of the source-code tree.
|
|
|
|
|
|
|
|
.. _`COPYING`:
|
|
|
|
https://github.com/ceph/ceph/blob/master/COPYING
|
|
|
|
|
|
|
|
Source code repositories
|
|
|
|
------------------------
|
|
|
|
|
|
|
|
The source code of Ceph lives on `GitHub`_ in a number of repositories below
|
|
|
|
the `Ceph "organization"`_.
|
|
|
|
|
|
|
|
.. _`Ceph "organization"`: https://github.com/ceph
|
|
|
|
|
2020-11-05 23:22:03 +00:00
|
|
|
A working knowledge of git_ is essential to make a meaningful contribution to the project as a developer.
|
2020-01-06 07:40:50 +00:00
|
|
|
|
|
|
|
.. _git: https://git-scm.com/doc
|
|
|
|
|
|
|
|
Although the `Ceph "organization"`_ includes several software repositories,
|
|
|
|
this document covers only one: https://github.com/ceph/ceph.
|
|
|
|
|
|
|
|
Redmine issue tracker
|
|
|
|
---------------------
|
|
|
|
|
|
|
|
Although `GitHub`_ is used for code, Ceph-related issues (Bugs, Features,
|
|
|
|
Backports, Documentation, etc.) are tracked at http://tracker.ceph.com,
|
|
|
|
which is powered by `Redmine`_.
|
|
|
|
|
|
|
|
.. _Redmine: http://www.redmine.org
|
|
|
|
|
|
|
|
The tracker has a Ceph project with a number of subprojects loosely
|
|
|
|
corresponding to the various architectural components (see
|
|
|
|
:doc:`/architecture`).
|
|
|
|
|
|
|
|
Mere `registration`_ in the tracker automatically grants permissions
|
|
|
|
sufficient to open new issues and comment on existing ones.
|
|
|
|
|
|
|
|
.. _registration: http://tracker.ceph.com/account/register
|
|
|
|
|
|
|
|
To report a bug or propose a new feature, `jump to the Ceph project`_ and
|
|
|
|
click on `New issue`_.
|
|
|
|
|
|
|
|
.. _`jump to the Ceph project`: http://tracker.ceph.com/projects/ceph
|
|
|
|
.. _`New issue`: http://tracker.ceph.com/projects/ceph/issues/new
|
|
|
|
|
2020-09-04 16:12:51 +00:00
|
|
|
.. _mailing-list:
|
|
|
|
|
2020-11-05 23:22:03 +00:00
|
|
|
Mailing lists
|
|
|
|
-------------
|
2020-01-06 07:40:50 +00:00
|
|
|
|
2020-11-05 23:22:03 +00:00
|
|
|
Ceph Development Mailing List
|
2021-07-09 21:32:45 +00:00
|
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
2020-01-06 07:40:50 +00:00
|
|
|
The ``dev@ceph.io`` list is for discussion about the development of Ceph,
|
|
|
|
its interoperability with other technology, and the operations of the
|
|
|
|
project itself.
|
|
|
|
|
2020-07-06 21:10:55 +00:00
|
|
|
The email discussion list for Ceph development is open to all. Subscribe by
|
|
|
|
sending a message to ``dev-request@ceph.io`` with the following line in the
|
2020-11-05 23:22:03 +00:00
|
|
|
body of the message::
|
2020-01-06 07:40:50 +00:00
|
|
|
|
|
|
|
subscribe ceph-devel
|
|
|
|
|
2020-11-05 23:22:03 +00:00
|
|
|
|
|
|
|
Ceph Client Patch Review Mailing List
|
2021-07-09 21:32:45 +00:00
|
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
2020-01-06 07:40:50 +00:00
|
|
|
The ``ceph-devel@vger.kernel.org`` list is for discussion and patch review
|
|
|
|
for the Linux kernel Ceph client component. Note that this list used to
|
2020-11-05 23:22:03 +00:00
|
|
|
be an all-encompassing list for developers. When searching the archives,
|
|
|
|
remember that this list contains the generic devel-ceph archives before mid-2018.
|
2020-01-06 07:40:50 +00:00
|
|
|
|
2020-11-05 23:22:03 +00:00
|
|
|
Subscribe to the list covering the Linux kernel Ceph client component by sending
|
|
|
|
a message to ``majordomo@vger.kernel.org`` with the following line in the body
|
|
|
|
of the message::
|
2020-01-06 07:40:50 +00:00
|
|
|
|
|
|
|
subscribe ceph-devel
|
|
|
|
|
|
|
|
|
2020-11-05 23:22:03 +00:00
|
|
|
Other Ceph Mailing Lists
|
2021-07-09 21:32:45 +00:00
|
|
|
~~~~~~~~~~~~~~~~~~~~~~~~
|
2020-01-06 07:40:50 +00:00
|
|
|
|
|
|
|
There are also `other Ceph-related mailing lists`_.
|
|
|
|
|
|
|
|
.. _`other Ceph-related mailing lists`: https://ceph.com/irc/
|
|
|
|
|
2020-09-04 16:12:51 +00:00
|
|
|
.. _irc:
|
2020-01-06 07:40:50 +00:00
|
|
|
|
2020-11-05 23:22:03 +00:00
|
|
|
|
2020-01-06 07:40:50 +00:00
|
|
|
IRC
|
|
|
|
---
|
|
|
|
|
2020-11-05 23:22:03 +00:00
|
|
|
In addition to mailing lists, the Ceph community also communicates in real time
|
|
|
|
using `Internet Relay Chat`_.
|
2020-01-06 07:40:50 +00:00
|
|
|
|
|
|
|
.. _`Internet Relay Chat`: http://www.irchelp.org/
|
|
|
|
|
2021-07-09 21:32:45 +00:00
|
|
|
The Ceph community gathers in the #ceph channel of the Open and Free Technology
|
|
|
|
Community (OFTC) IRC network.
|
|
|
|
|
|
|
|
Created in 1988, Internet Relay Chat (IRC) is a relay-based, real-time chat
|
|
|
|
protocol. It is mainly designed for group (many-to-many) communication in
|
|
|
|
discussion forums called channels, but also allows one-to-one communication via
|
|
|
|
private message. On IRC you can talk to many other members using Ceph, on
|
|
|
|
topics ranging from idle chit-chat to support questions. Though a channel might
|
|
|
|
have many people in it at any one time, they might not always be at their
|
|
|
|
keyboard; so if no-one responds, just wait around and someone will hopefully
|
|
|
|
answer soon enough.
|
|
|
|
|
|
|
|
Registration
|
|
|
|
~~~~~~~~~~~~
|
|
|
|
|
|
|
|
If you intend to use the IRC service on a continued basis, you are advised to
|
|
|
|
register an account. Registering gives you a unique IRC identity and allows you
|
|
|
|
to access channels where unregistered users have been locked out for technical
|
|
|
|
reasons.
|
|
|
|
|
|
|
|
|
|
|
|
Channels
|
|
|
|
~~~~~~~~
|
|
|
|
|
|
|
|
To connect to the OFTC IRC network, download an IRC client and configure it to
|
|
|
|
connect to ``irc.oftc.net``. Then join one or more of the channels. Discussions
|
|
|
|
inside #ceph are logged and archives are available online.
|
|
|
|
|
|
|
|
Here are the real-time discussion channels for the Ceph community:
|
|
|
|
|
|
|
|
- #ceph
|
|
|
|
- #ceph-devel
|
|
|
|
- #cephfs
|
|
|
|
- #ceph-dashboard
|
|
|
|
- #ceph-orchestrators
|
|
|
|
- #sepia
|
|
|
|
|
2020-01-06 07:40:50 +00:00
|
|
|
|
2020-09-04 16:12:51 +00:00
|
|
|
.. _submitting-patches:
|
|
|
|
|
2020-01-06 07:40:50 +00:00
|
|
|
Submitting patches
|
|
|
|
------------------
|
|
|
|
|
|
|
|
The canonical instructions for submitting patches are contained in the
|
|
|
|
file `CONTRIBUTING.rst`_ in the top-level directory of the source-code
|
|
|
|
tree. There may be some overlap between this guide and that file.
|
|
|
|
|
|
|
|
.. _`CONTRIBUTING.rst`:
|
2022-06-13 21:48:46 +00:00
|
|
|
https://github.com/ceph/ceph/blob/main/CONTRIBUTING.rst
|
2020-01-06 07:40:50 +00:00
|
|
|
|
|
|
|
All newcomers are encouraged to read that file carefully.
|
|
|
|
|
|
|
|
Building from source
|
|
|
|
--------------------
|
|
|
|
|
|
|
|
See instructions at :doc:`/install/build-ceph`.
|
|
|
|
|
|
|
|
Using ccache to speed up local builds
|
|
|
|
-------------------------------------
|
2020-11-05 23:22:03 +00:00
|
|
|
`ccache`_ can make the process of rebuilding the ceph source tree faster.
|
|
|
|
|
|
|
|
Before you use `ccache`_ to speed up your rebuilds of the ceph source tree,
|
|
|
|
make sure that your source tree is clean and will produce no build failures.
|
|
|
|
When you have a clean source tree, you can confidently use `ccache`_, secure in
|
|
|
|
the knowledge that you're not using a dirty tree.
|
|
|
|
|
|
|
|
Old build artifacts can cause build failures. You might introduce these
|
|
|
|
artifacts unknowingly when switching from one branch to another. If you see
|
|
|
|
build errors when you attempt a local build, follow the procedure below to
|
|
|
|
clean your source tree.
|
2020-01-06 07:40:50 +00:00
|
|
|
|
2020-11-05 23:22:03 +00:00
|
|
|
Cleaning the Source Tree
|
2021-07-09 21:32:45 +00:00
|
|
|
~~~~~~~~~~~~~~~~~~~~~~~~
|
2020-09-04 11:41:30 +00:00
|
|
|
|
2020-11-05 23:22:03 +00:00
|
|
|
.. prompt:: bash $
|
|
|
|
|
2021-04-06 13:58:13 +00:00
|
|
|
ninja clean
|
2020-11-05 23:22:03 +00:00
|
|
|
|
2020-11-13 07:37:58 +00:00
|
|
|
.. note:: The following commands will remove everything in the source tree
|
|
|
|
that isn't tracked by git. Make sure to back up your log files
|
|
|
|
and configuration options before running these commands.
|
2020-01-06 07:40:50 +00:00
|
|
|
|
2020-11-05 23:22:03 +00:00
|
|
|
.. prompt:: bash $
|
2020-01-06 07:40:50 +00:00
|
|
|
|
2020-11-05 23:22:03 +00:00
|
|
|
git clean -fdx; git submodule foreach git clean -fdx
|
2020-01-06 07:40:50 +00:00
|
|
|
|
2020-11-05 23:22:03 +00:00
|
|
|
Building Ceph with ccache
|
2021-07-09 21:32:45 +00:00
|
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
2020-11-05 23:22:03 +00:00
|
|
|
``ccache`` is available as a package in most distros. To build ceph with
|
|
|
|
ccache, run the following command.
|
2020-01-06 07:40:50 +00:00
|
|
|
|
2020-09-04 11:41:30 +00:00
|
|
|
.. prompt:: bash $
|
|
|
|
|
|
|
|
cmake -DWITH_CCACHE=ON ..
|
2020-01-06 07:40:50 +00:00
|
|
|
|
2020-11-05 23:22:03 +00:00
|
|
|
Using ccache to Speed Up Build Times
|
2021-07-09 21:32:45 +00:00
|
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
2020-11-05 23:22:03 +00:00
|
|
|
``ccache`` can be used for speeding up all builds of the system. For more
|
|
|
|
details, refer to the `run modes`_ section of the ccache manual. The default
|
|
|
|
settings of ``ccache`` can be displayed with the ``ccache -s`` command.
|
2020-01-06 07:40:50 +00:00
|
|
|
|
2020-11-05 23:22:03 +00:00
|
|
|
.. note:: We recommend overriding the ``max_size``. The default is 10G.
|
|
|
|
Use a larger value, like 25G. Refer to the `configuration`_ section
|
|
|
|
of the ccache manual for more information.
|
2020-01-06 07:40:50 +00:00
|
|
|
|
|
|
|
To further increase the cache hit rate and reduce compile times in a
|
2020-11-05 23:22:03 +00:00
|
|
|
development environment, set the version information and build timestamps to
|
|
|
|
fixed values. This makes it unnecessary to rebuild the binaries that contain
|
|
|
|
this information.
|
2020-01-06 07:40:50 +00:00
|
|
|
|
|
|
|
This can be achieved by adding the following settings to the ``ccache``
|
|
|
|
configuration file ``ccache.conf``::
|
|
|
|
|
|
|
|
sloppiness = time_macros
|
|
|
|
run_second_cpp = true
|
|
|
|
|
|
|
|
Now, set the environment variable ``SOURCE_DATE_EPOCH`` to a fixed value (a
|
|
|
|
UNIX timestamp) and set ``ENABLE_GIT_VERSION`` to ``OFF`` when running
|
2020-11-13 07:37:58 +00:00
|
|
|
``cmake``:
|
2020-09-04 11:41:30 +00:00
|
|
|
|
|
|
|
.. prompt:: bash $
|
2020-01-06 07:40:50 +00:00
|
|
|
|
2020-09-04 11:41:30 +00:00
|
|
|
export SOURCE_DATE_EPOCH=946684800
|
|
|
|
cmake -DWITH_CCACHE=ON -DENABLE_GIT_VERSION=OFF ..
|
2020-01-06 07:40:50 +00:00
|
|
|
|
|
|
|
.. note:: Binaries produced with these build options are not suitable for
|
|
|
|
production or debugging purposes, as they do not contain the correct build
|
|
|
|
time and git version information.
|
|
|
|
|
|
|
|
.. _`ccache`: https://ccache.samba.org/
|
|
|
|
.. _`run modes`: https://ccache.samba.org/manual.html#_run_modes
|
|
|
|
.. _`configuration`: https://ccache.samba.org/manual.html#_configuration
|
|
|
|
|
|
|
|
Development-mode cluster
|
|
|
|
------------------------
|
|
|
|
|
|
|
|
See :doc:`/dev/quick_guide`.
|
|
|
|
|
|
|
|
Kubernetes/Rook development cluster
|
|
|
|
-----------------------------------
|
|
|
|
|
|
|
|
See :ref:`kubernetes-dev`
|
|
|
|
|
2020-09-04 16:12:51 +00:00
|
|
|
.. _backporting:
|
|
|
|
|
2020-01-06 07:40:50 +00:00
|
|
|
Backporting
|
|
|
|
-----------
|
|
|
|
|
2022-06-13 21:48:46 +00:00
|
|
|
All bugfixes should be merged to the ``main`` branch before being
|
2020-01-06 07:40:50 +00:00
|
|
|
backported. To flag a bugfix for backporting, make sure it has a
|
|
|
|
`tracker issue`_ associated with it and set the ``Backport`` field to a
|
|
|
|
comma-separated list of previous releases (e.g. "hammer,jewel") that you think
|
|
|
|
need the backport.
|
|
|
|
The rest (including the actual backporting) will be taken care of by the
|
|
|
|
`Stable Releases and Backports`_ team.
|
|
|
|
|
|
|
|
.. _`tracker issue`: http://tracker.ceph.com/
|
|
|
|
.. _`Stable Releases and Backports`: http://tracker.ceph.com/projects/ceph-releases/wiki
|
|
|
|
|
|
|
|
Guidance for use of cluster log
|
|
|
|
-------------------------------
|
|
|
|
|
|
|
|
If your patches emit messages to the Ceph cluster log, please consult
|
2020-07-13 14:19:19 +00:00
|
|
|
this: :doc:`/dev/logging`.
|