libabigail/include/abg-workers.h

118 lines
3.4 KiB
C
Raw Normal View History

Use worker threads pattern to speed up some tests We are going to need to speed up more and more tests, and coding directly with libpthread for that can be tedious and bug-prone. So I devised an implementation for the worker threads design pattern instead, and used it to speed up some tests. * include/Makefile.am: Add the new abg-workers.h to source distribution. * include/abg-workers.h: New file. * src/Makefile.am: Add the new abg-worker.cc to source distribution. * src/abg-workers.cc: New file. * tests/test-utils.cc: Update copyright. Make get_src_dir() and get_build_dir() return a const char*, as opposed to returning a string. Make that const char reside in thread local storage, so that two concurrent threads can safely call these functions in parallel, without any race. * tests/test-utils.h: Make get_src_dir() and get_build_dir() return a const char*, as opposed to returning a string. * tests/test-abicompat.cc: Update copyright. Adjust for get_src_dir() and get_build_dir() change. * tests/test-abidiff.cc: Likewise. * tests/test-alt-dwarf-file.cc: Likewise. * tests/test-core-diff.cc: Likewise. * tests/test-diff-dwarf-abixml.cc: Likewise. * tests/test-diff-dwarf.cc: Likewise. * tests/test-diff-pkg.cc: Likewise. * tests/test-diff-suppr.cc: Likewise. * tests/test-lookup-syms.cc: Likewise. * tests/test-read-dwarf.cc: Likewise. * tests/test-read-write.cc: Likewise. * tests/test-types-stability.cc: Likewise. Use the new task queue type to run these tests in parallel. * tests/test-diff-filter.cc: Likewise. Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-01-15 12:34:16 +00:00
// -*- Mode: C++ -*-
//
Update copyright year to 2020 We are in February 2020 so this is long overdue. * include/abg-comp-filter.h: Update copyright year to 2020. * include/abg-comparison.h: Likewise. * include/abg-config.h: Likewise. * include/abg-corpus.h: Likewise. * include/abg-cxx-compat.h: Likewise. * include/abg-diff-utils.h: Likewise. * include/abg-dwarf-reader.h: Likewise. * include/abg-fwd.h: Likewise. * include/abg-hash.h: Likewise. * include/abg-ini.h: Likewise. * include/abg-interned-str.h: Likewise. * include/abg-ir.h: Likewise. * include/abg-libxml-utils.h: Likewise. * include/abg-libzip-utils.h: Likewise. * include/abg-reader.h: Likewise. * include/abg-reporter.h: Likewise. * include/abg-sptr-utils.h: Likewise. * include/abg-suppression.h: Likewise. * include/abg-tools-utils.h: Likewise. * include/abg-traverse.h: Likewise. * include/abg-viz-common.h: Likewise. * include/abg-viz-dot.h: Likewise. * include/abg-viz-svg.h: Likewise. * include/abg-workers.h: Likewise. * include/abg-writer.h: Likewise. * src/abg-comp-filter.cc: Likewise. * src/abg-comparison-priv.h: Likewise. * src/abg-comparison.cc: Likewise. * src/abg-config.cc: Likewise. * src/abg-corpus-priv.h: Likewise. * src/abg-corpus.cc: Likewise. * src/abg-default-reporter.cc: Likewise. * src/abg-diff-utils.cc: Likewise. * src/abg-dwarf-reader.cc: Likewise. * src/abg-hash.cc: Likewise. * src/abg-ini.cc: Likewise. * src/abg-internal.h: Likewise. * src/abg-ir-priv.h: Likewise. * src/abg-ir.cc: Likewise. * src/abg-leaf-reporter.cc: Likewise. * src/abg-libxml-utils.cc: Likewise. * src/abg-libzip-utils.cc: Likewise. * src/abg-reader.cc: Likewise. * src/abg-reporter-priv.cc: Likewise. * src/abg-reporter-priv.h: Likewise. * src/abg-sptr-utils.cc: Likewise. * src/abg-suppression-priv.h: Likewise. * src/abg-suppression.cc: Likewise. * src/abg-tools-utils.cc: Likewise. * src/abg-traverse.cc: Likewise. * src/abg-viz-common.cc: Likewise. * src/abg-viz-dot.cc: Likewise. * src/abg-viz-svg.cc: Likewise. * src/abg-workers.cc: Likewise. * src/abg-writer.cc: Likewise. * tests/print-diff-tree.cc: Likewise. * tests/test-abicompat.cc: Likewise. * tests/test-abidiff-exit.cc: Likewise. * tests/test-abidiff.cc: Likewise. * tests/test-alt-dwarf-file.cc: Likewise. * tests/test-core-diff.cc: Likewise. * tests/test-diff-dwarf-abixml.cc: Likewise. * tests/test-diff-dwarf.cc: Likewise. * tests/test-diff-filter.cc: Likewise. * tests/test-diff-pkg.cc: Likewise. * tests/test-diff-suppr.cc: Likewise. * tests/test-diff2.cc: Likewise. * tests/test-dot.cc: Likewise. * tests/test-ini.cc: Likewise. * tests/test-ir-walker.cc: Likewise. * tests/test-lookup-syms.cc: Likewise. * tests/test-read-dwarf.cc: Likewise. * tests/test-read-write.cc: Likewise. * tests/test-svg.cc: Likewise. * tests/test-tools-utils.cc: Likewise. * tests/test-types-stability.cc: Likewise. * tests/test-utils.cc: Likewise. * tests/test-utils.h: Likewise. * tests/test-write-read-archive.cc: Likewise. * tools/abiar.cc: Likewise. * tools/abicompat.cc: Likewise. * tools/abidiff.cc: Likewise. * tools/abidw.cc: Likewise. * tools/abilint.cc: Likewise. * tools/abipkgdiff.cc: Likewise. * tools/abisym.cc: Likewise. * tools/binilint.cc: Likewise. * tools/kmidiff.cc: Likewise. * update-copyright.sh: Fix the updating script to handle not just "Red Hat, Inc." Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2020-02-21 15:42:35 +00:00
// Copyright (C) 2013-2020 Red Hat, Inc.
Use worker threads pattern to speed up some tests We are going to need to speed up more and more tests, and coding directly with libpthread for that can be tedious and bug-prone. So I devised an implementation for the worker threads design pattern instead, and used it to speed up some tests. * include/Makefile.am: Add the new abg-workers.h to source distribution. * include/abg-workers.h: New file. * src/Makefile.am: Add the new abg-worker.cc to source distribution. * src/abg-workers.cc: New file. * tests/test-utils.cc: Update copyright. Make get_src_dir() and get_build_dir() return a const char*, as opposed to returning a string. Make that const char reside in thread local storage, so that two concurrent threads can safely call these functions in parallel, without any race. * tests/test-utils.h: Make get_src_dir() and get_build_dir() return a const char*, as opposed to returning a string. * tests/test-abicompat.cc: Update copyright. Adjust for get_src_dir() and get_build_dir() change. * tests/test-abidiff.cc: Likewise. * tests/test-alt-dwarf-file.cc: Likewise. * tests/test-core-diff.cc: Likewise. * tests/test-diff-dwarf-abixml.cc: Likewise. * tests/test-diff-dwarf.cc: Likewise. * tests/test-diff-pkg.cc: Likewise. * tests/test-diff-suppr.cc: Likewise. * tests/test-lookup-syms.cc: Likewise. * tests/test-read-dwarf.cc: Likewise. * tests/test-read-write.cc: Likewise. * tests/test-types-stability.cc: Likewise. Use the new task queue type to run these tests in parallel. * tests/test-diff-filter.cc: Likewise. Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-01-15 12:34:16 +00:00
//
// This file is part of the GNU Application Binary Interface Generic
// Analysis and Instrumentation Library (libabigail). This library is
// free software; you can redistribute it and/or modify it under the
// terms of the GNU Lesser General Public License as published by the
// Free Software Foundation; either version 3, or (at your option) any
// later version.
// This library is distributed in the hope that it will be useful, but
// WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
// General Lesser Public License for more details.
// You should have received a copy of the GNU Lesser General Public
// License along with this program; see the file COPYING-LGPLV3. If
// not, see <http://www.gnu.org/licenses/>.
// Author: Dodji Seketeli
/// @file
///
/// This file declares an interface for the worker threads (or thread
/// pool) design pattern. It aims at performing a set of tasks in
/// parallel, using the multi-threading capabilities of the underlying
/// processor(s).
///
#ifndef __ABG_WORKERS_H__
#define __ABG_WORKERS_H__
#include <vector>
Add compatibility layer for C++11 mode Introduce a compatibility layer for C++11 code by adding include/abg-cxx-compat.h. abg-cxx-compat defines a new namespace abg_compat and defines abg_compat::hash abg_compat::shared_ptr abg_compat::weak_ptr abg_compat::dynamic_pointer_cast abg_compat::static_pointer_cast abg_compat::unordered_map abg_compat::unordered_set based on definitions from std::tr1 (std=gnu++98) or std:: (std=gnu++11). I decided for introducing abg_compat:: rather than polluting abigail:: to allow an easier transition to C++11 at a later time and to not subtly break existing code. As the shared_ptr in C++11 defines shared_ptr::operator bool() explicit, some locations where a shared_ptr is assigned to boolean, needed to be adjusted to explicitly cast to bool. * include/abg-cxx-compat.h: new file introducing the abg_compat namespace to provide C++11 functionality from either std::tr1 or std:: * include/Makefile.am: Add the new abg-cxx-compat.h to source distribution. * include/abg-comparison.h: replace std::tr1 usage by abg_compat and adjust includes accordingly: likewise * include/abg-diff-utils.h: likewise * include/abg-fwd.h: likewise * include/abg-ini.h: likewise * include/abg-interned-str.h: likewise * include/abg-ir.h: likewise * include/abg-libxml-utils.h: likewise * include/abg-libzip-utils.h: likewise * include/abg-reporter.h: likewise * include/abg-sptr-utils.h: likewise * include/abg-suppression.h: likewise * include/abg-tools-utils.h: likewise * include/abg-workers.h: likewise * src/abg-comp-filter.cc: likewise * src/abg-comparison-priv.h: likewise * src/abg-corpus.cc: likewise * src/abg-dwarf-reader.cc: likewise * src/abg-hash.cc: likewise * src/abg-ir.cc: likewise * src/abg-reader.cc: likewise * src/abg-suppression.cc: likewise * src/abg-tools-utils.cc: likewise * src/abg-writer.cc: likewise * tests/test-diff-filter.cc: likewise * tests/test-diff-pkg.cc: likewise * tests/test-read-dwarf.cc: likewise * tests/test-read-write.cc: likewise * tests/test-types-stability.cc: likewise * tests/test-write-read-archive.cc: likewise * tools/abicompat.cc: likewise * tools/abidiff.cc: likewise * tools/abidw.cc: likewise * tools/abilint.cc: likewise * tools/abipkgdiff.cc: likewise Signed-off-by: Matthias Maennich <maennich@google.com> Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2019-07-05 11:00:23 +00:00
#include "abg-cxx-compat.h"
Use worker threads pattern to speed up some tests We are going to need to speed up more and more tests, and coding directly with libpthread for that can be tedious and bug-prone. So I devised an implementation for the worker threads design pattern instead, and used it to speed up some tests. * include/Makefile.am: Add the new abg-workers.h to source distribution. * include/abg-workers.h: New file. * src/Makefile.am: Add the new abg-worker.cc to source distribution. * src/abg-workers.cc: New file. * tests/test-utils.cc: Update copyright. Make get_src_dir() and get_build_dir() return a const char*, as opposed to returning a string. Make that const char reside in thread local storage, so that two concurrent threads can safely call these functions in parallel, without any race. * tests/test-utils.h: Make get_src_dir() and get_build_dir() return a const char*, as opposed to returning a string. * tests/test-abicompat.cc: Update copyright. Adjust for get_src_dir() and get_build_dir() change. * tests/test-abidiff.cc: Likewise. * tests/test-alt-dwarf-file.cc: Likewise. * tests/test-core-diff.cc: Likewise. * tests/test-diff-dwarf-abixml.cc: Likewise. * tests/test-diff-dwarf.cc: Likewise. * tests/test-diff-pkg.cc: Likewise. * tests/test-diff-suppr.cc: Likewise. * tests/test-lookup-syms.cc: Likewise. * tests/test-read-dwarf.cc: Likewise. * tests/test-read-write.cc: Likewise. * tests/test-types-stability.cc: Likewise. Use the new task queue type to run these tests in parallel. * tests/test-diff-filter.cc: Likewise. Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-01-15 12:34:16 +00:00
Add compatibility layer for C++11 mode Introduce a compatibility layer for C++11 code by adding include/abg-cxx-compat.h. abg-cxx-compat defines a new namespace abg_compat and defines abg_compat::hash abg_compat::shared_ptr abg_compat::weak_ptr abg_compat::dynamic_pointer_cast abg_compat::static_pointer_cast abg_compat::unordered_map abg_compat::unordered_set based on definitions from std::tr1 (std=gnu++98) or std:: (std=gnu++11). I decided for introducing abg_compat:: rather than polluting abigail:: to allow an easier transition to C++11 at a later time and to not subtly break existing code. As the shared_ptr in C++11 defines shared_ptr::operator bool() explicit, some locations where a shared_ptr is assigned to boolean, needed to be adjusted to explicitly cast to bool. * include/abg-cxx-compat.h: new file introducing the abg_compat namespace to provide C++11 functionality from either std::tr1 or std:: * include/Makefile.am: Add the new abg-cxx-compat.h to source distribution. * include/abg-comparison.h: replace std::tr1 usage by abg_compat and adjust includes accordingly: likewise * include/abg-diff-utils.h: likewise * include/abg-fwd.h: likewise * include/abg-ini.h: likewise * include/abg-interned-str.h: likewise * include/abg-ir.h: likewise * include/abg-libxml-utils.h: likewise * include/abg-libzip-utils.h: likewise * include/abg-reporter.h: likewise * include/abg-sptr-utils.h: likewise * include/abg-suppression.h: likewise * include/abg-tools-utils.h: likewise * include/abg-workers.h: likewise * src/abg-comp-filter.cc: likewise * src/abg-comparison-priv.h: likewise * src/abg-corpus.cc: likewise * src/abg-dwarf-reader.cc: likewise * src/abg-hash.cc: likewise * src/abg-ir.cc: likewise * src/abg-reader.cc: likewise * src/abg-suppression.cc: likewise * src/abg-tools-utils.cc: likewise * src/abg-writer.cc: likewise * tests/test-diff-filter.cc: likewise * tests/test-diff-pkg.cc: likewise * tests/test-read-dwarf.cc: likewise * tests/test-read-write.cc: likewise * tests/test-types-stability.cc: likewise * tests/test-write-read-archive.cc: likewise * tools/abicompat.cc: likewise * tools/abidiff.cc: likewise * tools/abidw.cc: likewise * tools/abilint.cc: likewise * tools/abipkgdiff.cc: likewise Signed-off-by: Matthias Maennich <maennich@google.com> Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2019-07-05 11:00:23 +00:00
using abg_compat::shared_ptr;
Use worker threads pattern to speed up some tests We are going to need to speed up more and more tests, and coding directly with libpthread for that can be tedious and bug-prone. So I devised an implementation for the worker threads design pattern instead, and used it to speed up some tests. * include/Makefile.am: Add the new abg-workers.h to source distribution. * include/abg-workers.h: New file. * src/Makefile.am: Add the new abg-worker.cc to source distribution. * src/abg-workers.cc: New file. * tests/test-utils.cc: Update copyright. Make get_src_dir() and get_build_dir() return a const char*, as opposed to returning a string. Make that const char reside in thread local storage, so that two concurrent threads can safely call these functions in parallel, without any race. * tests/test-utils.h: Make get_src_dir() and get_build_dir() return a const char*, as opposed to returning a string. * tests/test-abicompat.cc: Update copyright. Adjust for get_src_dir() and get_build_dir() change. * tests/test-abidiff.cc: Likewise. * tests/test-alt-dwarf-file.cc: Likewise. * tests/test-core-diff.cc: Likewise. * tests/test-diff-dwarf-abixml.cc: Likewise. * tests/test-diff-dwarf.cc: Likewise. * tests/test-diff-pkg.cc: Likewise. * tests/test-diff-suppr.cc: Likewise. * tests/test-lookup-syms.cc: Likewise. * tests/test-read-dwarf.cc: Likewise. * tests/test-read-write.cc: Likewise. * tests/test-types-stability.cc: Likewise. Use the new task queue type to run these tests in parallel. * tests/test-diff-filter.cc: Likewise. Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-01-15 12:34:16 +00:00
namespace abigail
{
/// The namespace of the worker threads (or thread pool)
/// implementation of libabigail. This was modelled after the article
/// https://en.wikipedia.org/wiki/Thread_pool.
Use worker threads pattern to speed up some tests We are going to need to speed up more and more tests, and coding directly with libpthread for that can be tedious and bug-prone. So I devised an implementation for the worker threads design pattern instead, and used it to speed up some tests. * include/Makefile.am: Add the new abg-workers.h to source distribution. * include/abg-workers.h: New file. * src/Makefile.am: Add the new abg-worker.cc to source distribution. * src/abg-workers.cc: New file. * tests/test-utils.cc: Update copyright. Make get_src_dir() and get_build_dir() return a const char*, as opposed to returning a string. Make that const char reside in thread local storage, so that two concurrent threads can safely call these functions in parallel, without any race. * tests/test-utils.h: Make get_src_dir() and get_build_dir() return a const char*, as opposed to returning a string. * tests/test-abicompat.cc: Update copyright. Adjust for get_src_dir() and get_build_dir() change. * tests/test-abidiff.cc: Likewise. * tests/test-alt-dwarf-file.cc: Likewise. * tests/test-core-diff.cc: Likewise. * tests/test-diff-dwarf-abixml.cc: Likewise. * tests/test-diff-dwarf.cc: Likewise. * tests/test-diff-pkg.cc: Likewise. * tests/test-diff-suppr.cc: Likewise. * tests/test-lookup-syms.cc: Likewise. * tests/test-read-dwarf.cc: Likewise. * tests/test-read-write.cc: Likewise. * tests/test-types-stability.cc: Likewise. Use the new task queue type to run these tests in parallel. * tests/test-diff-filter.cc: Likewise. Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-01-15 12:34:16 +00:00
namespace workers
{
size_t get_number_of_threads();
/// This represents a task to be performed.
///
/// Each instance of this type represents a task that can be performed
/// concurrently to other instance of the same type.
///
/// An instance of @ref task is meant to be performed by a worker
/// (thread). A set of tasks can be stored in a @ref queue.
Use worker threads pattern to speed up some tests We are going to need to speed up more and more tests, and coding directly with libpthread for that can be tedious and bug-prone. So I devised an implementation for the worker threads design pattern instead, and used it to speed up some tests. * include/Makefile.am: Add the new abg-workers.h to source distribution. * include/abg-workers.h: New file. * src/Makefile.am: Add the new abg-worker.cc to source distribution. * src/abg-workers.cc: New file. * tests/test-utils.cc: Update copyright. Make get_src_dir() and get_build_dir() return a const char*, as opposed to returning a string. Make that const char reside in thread local storage, so that two concurrent threads can safely call these functions in parallel, without any race. * tests/test-utils.h: Make get_src_dir() and get_build_dir() return a const char*, as opposed to returning a string. * tests/test-abicompat.cc: Update copyright. Adjust for get_src_dir() and get_build_dir() change. * tests/test-abidiff.cc: Likewise. * tests/test-alt-dwarf-file.cc: Likewise. * tests/test-core-diff.cc: Likewise. * tests/test-diff-dwarf-abixml.cc: Likewise. * tests/test-diff-dwarf.cc: Likewise. * tests/test-diff-pkg.cc: Likewise. * tests/test-diff-suppr.cc: Likewise. * tests/test-lookup-syms.cc: Likewise. * tests/test-read-dwarf.cc: Likewise. * tests/test-read-write.cc: Likewise. * tests/test-types-stability.cc: Likewise. Use the new task queue type to run these tests in parallel. * tests/test-diff-filter.cc: Likewise. Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-01-15 12:34:16 +00:00
class task
{
public:
virtual void
perform() = 0;
abg-workers: Rework the worker queue to improve concurrent behaviour This patch refactors the abigail::workers::queue and abigail::workers::worker implementations to avoid holding locking primitives longer than necessary. In particular, the queue_cond_mutex was held during the entiry worker runtime, effectively serializing the workers. Hence, use a mutex+cond pair for each, the input and output queue and only synchronize around the interaction with their corresponding queues. The tasks_todo_(mutex|cond) are meant to synchronize scheduling and distribution of work among workers, while tasks_done_(mutex|cond) are used for synchronizing threads when putting back the tasks to the output queue and to hold back threads waiting for the queue and workers to drain. Along that way, I did some cleanup that was now possible. - Move entire implementation of abigail::workers::task into header. - Make default_notify a static member. - Replace the multiple constructors with one with default arguments. * include/abg-workers.h (workers::task): move entire implementation to header and drop superfluous forward declaration. * src/abg-workers.cc (workers::task):: Likewise. (workers::queue::priv): Drop queue_cond_mutex, rename queue_cond to tasks_todo_cond, add task_done_cond, make default_notify static. (workers::queue::priv::priv): Add default arguments to fully qualified constructor, drop the remaining ones. (workers::queue:prive::more_tasks_to_execute): Drop method. (workers::queue:prive::schedule_task): Do not synchronize access to the queue condition variable, but only on the mutex. (do_bring_workers_down): Likewise. Also await tasks_done to be empty. (workers::queue:prive::worker::wait_to_execute_a_task): Await tasks on the tasks_todo with tasks_todo_(cond|mutex) and signal task completion to tasks_done_cond. Signed-off-by: Matthias Maennich <maennich@google.com> Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2020-02-04 13:05:49 +00:00
virtual ~task(){};
Use worker threads pattern to speed up some tests We are going to need to speed up more and more tests, and coding directly with libpthread for that can be tedious and bug-prone. So I devised an implementation for the worker threads design pattern instead, and used it to speed up some tests. * include/Makefile.am: Add the new abg-workers.h to source distribution. * include/abg-workers.h: New file. * src/Makefile.am: Add the new abg-worker.cc to source distribution. * src/abg-workers.cc: New file. * tests/test-utils.cc: Update copyright. Make get_src_dir() and get_build_dir() return a const char*, as opposed to returning a string. Make that const char reside in thread local storage, so that two concurrent threads can safely call these functions in parallel, without any race. * tests/test-utils.h: Make get_src_dir() and get_build_dir() return a const char*, as opposed to returning a string. * tests/test-abicompat.cc: Update copyright. Adjust for get_src_dir() and get_build_dir() change. * tests/test-abidiff.cc: Likewise. * tests/test-alt-dwarf-file.cc: Likewise. * tests/test-core-diff.cc: Likewise. * tests/test-diff-dwarf-abixml.cc: Likewise. * tests/test-diff-dwarf.cc: Likewise. * tests/test-diff-pkg.cc: Likewise. * tests/test-diff-suppr.cc: Likewise. * tests/test-lookup-syms.cc: Likewise. * tests/test-read-dwarf.cc: Likewise. * tests/test-read-write.cc: Likewise. * tests/test-types-stability.cc: Likewise. Use the new task queue type to run these tests in parallel. * tests/test-diff-filter.cc: Likewise. Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-01-15 12:34:16 +00:00
}; // end class task.
abg-workers: Rework the worker queue to improve concurrent behaviour This patch refactors the abigail::workers::queue and abigail::workers::worker implementations to avoid holding locking primitives longer than necessary. In particular, the queue_cond_mutex was held during the entiry worker runtime, effectively serializing the workers. Hence, use a mutex+cond pair for each, the input and output queue and only synchronize around the interaction with their corresponding queues. The tasks_todo_(mutex|cond) are meant to synchronize scheduling and distribution of work among workers, while tasks_done_(mutex|cond) are used for synchronizing threads when putting back the tasks to the output queue and to hold back threads waiting for the queue and workers to drain. Along that way, I did some cleanup that was now possible. - Move entire implementation of abigail::workers::task into header. - Make default_notify a static member. - Replace the multiple constructors with one with default arguments. * include/abg-workers.h (workers::task): move entire implementation to header and drop superfluous forward declaration. * src/abg-workers.cc (workers::task):: Likewise. (workers::queue::priv): Drop queue_cond_mutex, rename queue_cond to tasks_todo_cond, add task_done_cond, make default_notify static. (workers::queue::priv::priv): Add default arguments to fully qualified constructor, drop the remaining ones. (workers::queue:prive::more_tasks_to_execute): Drop method. (workers::queue:prive::schedule_task): Do not synchronize access to the queue condition variable, but only on the mutex. (do_bring_workers_down): Likewise. Also await tasks_done to be empty. (workers::queue:prive::worker::wait_to_execute_a_task): Await tasks on the tasks_todo with tasks_todo_(cond|mutex) and signal task completion to tasks_done_cond. Signed-off-by: Matthias Maennich <maennich@google.com> Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2020-02-04 13:05:49 +00:00
typedef shared_ptr<task> task_sptr;
/// This represents a queue of tasks to be performed.
///
/// Tasks are performed by a number of worker threads.
///
/// When a task is inserted into a @ref queue, the task is said to be
/// "scheduled for execution".
///
/// This is because there are worker threads waiting for tasks to be
/// added to the queue. When a task is added to the queue, a worker
/// thread picks it up, executes it, notifies interested listeners
/// when the @ref task's execution is completed, and waits for another
/// task to be added to the queue.
///
/// Of course, several worker threads can execute tasks concurrently.
Use worker threads pattern to speed up some tests We are going to need to speed up more and more tests, and coding directly with libpthread for that can be tedious and bug-prone. So I devised an implementation for the worker threads design pattern instead, and used it to speed up some tests. * include/Makefile.am: Add the new abg-workers.h to source distribution. * include/abg-workers.h: New file. * src/Makefile.am: Add the new abg-worker.cc to source distribution. * src/abg-workers.cc: New file. * tests/test-utils.cc: Update copyright. Make get_src_dir() and get_build_dir() return a const char*, as opposed to returning a string. Make that const char reside in thread local storage, so that two concurrent threads can safely call these functions in parallel, without any race. * tests/test-utils.h: Make get_src_dir() and get_build_dir() return a const char*, as opposed to returning a string. * tests/test-abicompat.cc: Update copyright. Adjust for get_src_dir() and get_build_dir() change. * tests/test-abidiff.cc: Likewise. * tests/test-alt-dwarf-file.cc: Likewise. * tests/test-core-diff.cc: Likewise. * tests/test-diff-dwarf-abixml.cc: Likewise. * tests/test-diff-dwarf.cc: Likewise. * tests/test-diff-pkg.cc: Likewise. * tests/test-diff-suppr.cc: Likewise. * tests/test-lookup-syms.cc: Likewise. * tests/test-read-dwarf.cc: Likewise. * tests/test-read-write.cc: Likewise. * tests/test-types-stability.cc: Likewise. Use the new task queue type to run these tests in parallel. * tests/test-diff-filter.cc: Likewise. Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-01-15 12:34:16 +00:00
class queue
{
public:
struct priv;
typedef shared_ptr<priv> priv_sptr;
Several fixes and enhancements to abigail::workers While making abipkgdiff to use the abigail::workers API to do away with using pthreads directory, it appeared that the abigail::workers API needs fixes and enhancements. Fixes ===== * Don't try to schedule a task if the pointer to the task is nil * Fix a data race when bringing workers (of a queue) down * Always try to wake up all waiting threads when bringing down queue workers. * Fix a data race when accessing the queue condition variable * Fix a data race when notifying listeners about the end of the job performed by the task. Enhancements ============ * Pass the "task done" notifier by reference, to the worker queue. Without this, the worker queue needs to copy the "task done" notifier by value. This implies that user code needs to provide task done notifier instances that come with potentially complicated copy constructors. By passing it by reference and by just re-using the notifier from the user code, we do away with the need for copying altogether. This also fixes some latent copying bugs. * Add a workers::queue::schedule_tasks() method This allows user code to schedule a vector of tasks at once. * make workers::queue::get_completed_tasks() return a non-const vector This enables user code to sort the completed tasks as they wish. * include/abg-workers.h (queue::tasks_type): New typedef. (queue::queue): Pass task_done_notify by reference. (queue::schedule_tasks): Declare new member function. (queue::get_completed_tasks): Return non-const vector. * src/abg-workers.cc (queue::priv::default_notify): New data member. (queue::priv::notify): Make this data member be a reference. (queue::priv::priv): Initialize the notify data member to either the new default_notify (if no notifier is provided by the constructor) or to the notifier provided by the constructor. (queue::priv::schedule_task): Do not schedule a nil task. Update comment. (queue::priv::schedule_tasks): Add a new member function. (queue::priv::do_bring_workers_down): Update comment. Protect access to "bring_workers_down" with tasks_todo_mutex to prevent a data race. Call pthread_cond_broadcast on the queue_cond unconditionaly to prevent some worker threads to keep waiting for ever. Also, protect the access to the queue_cond by the queue_cond_mutex to precent a data race. (queue::queue): Pass the notifier by reference. Update comment. (queue::schedule_task): Update comment. (queue::schedule_tasks): Define new member function. (queue::wait_for_workers_to_complete): Update comment. (queue::get_completed_tasks): Return a non-const vector. Update comment. (worker::wait_to_execute_a_task): Update several comments. Make the execution of the notification code to be synchronized (on the tasks_done_mutex). Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2017-02-24 09:58:16 +00:00
/// A convenience typedef for a vector of @ref task_sptr
typedef std::vector<task_sptr> tasks_type;
Use worker threads pattern to speed up some tests We are going to need to speed up more and more tests, and coding directly with libpthread for that can be tedious and bug-prone. So I devised an implementation for the worker threads design pattern instead, and used it to speed up some tests. * include/Makefile.am: Add the new abg-workers.h to source distribution. * include/abg-workers.h: New file. * src/Makefile.am: Add the new abg-worker.cc to source distribution. * src/abg-workers.cc: New file. * tests/test-utils.cc: Update copyright. Make get_src_dir() and get_build_dir() return a const char*, as opposed to returning a string. Make that const char reside in thread local storage, so that two concurrent threads can safely call these functions in parallel, without any race. * tests/test-utils.h: Make get_src_dir() and get_build_dir() return a const char*, as opposed to returning a string. * tests/test-abicompat.cc: Update copyright. Adjust for get_src_dir() and get_build_dir() change. * tests/test-abidiff.cc: Likewise. * tests/test-alt-dwarf-file.cc: Likewise. * tests/test-core-diff.cc: Likewise. * tests/test-diff-dwarf-abixml.cc: Likewise. * tests/test-diff-dwarf.cc: Likewise. * tests/test-diff-pkg.cc: Likewise. * tests/test-diff-suppr.cc: Likewise. * tests/test-lookup-syms.cc: Likewise. * tests/test-read-dwarf.cc: Likewise. * tests/test-read-write.cc: Likewise. * tests/test-types-stability.cc: Likewise. Use the new task queue type to run these tests in parallel. * tests/test-diff-filter.cc: Likewise. Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-01-15 12:34:16 +00:00
private:
priv_sptr p_;
public:
struct task_done_notify;
queue();
queue(unsigned number_of_workers);
queue(unsigned number_of_workers,
Several fixes and enhancements to abigail::workers While making abipkgdiff to use the abigail::workers API to do away with using pthreads directory, it appeared that the abigail::workers API needs fixes and enhancements. Fixes ===== * Don't try to schedule a task if the pointer to the task is nil * Fix a data race when bringing workers (of a queue) down * Always try to wake up all waiting threads when bringing down queue workers. * Fix a data race when accessing the queue condition variable * Fix a data race when notifying listeners about the end of the job performed by the task. Enhancements ============ * Pass the "task done" notifier by reference, to the worker queue. Without this, the worker queue needs to copy the "task done" notifier by value. This implies that user code needs to provide task done notifier instances that come with potentially complicated copy constructors. By passing it by reference and by just re-using the notifier from the user code, we do away with the need for copying altogether. This also fixes some latent copying bugs. * Add a workers::queue::schedule_tasks() method This allows user code to schedule a vector of tasks at once. * make workers::queue::get_completed_tasks() return a non-const vector This enables user code to sort the completed tasks as they wish. * include/abg-workers.h (queue::tasks_type): New typedef. (queue::queue): Pass task_done_notify by reference. (queue::schedule_tasks): Declare new member function. (queue::get_completed_tasks): Return non-const vector. * src/abg-workers.cc (queue::priv::default_notify): New data member. (queue::priv::notify): Make this data member be a reference. (queue::priv::priv): Initialize the notify data member to either the new default_notify (if no notifier is provided by the constructor) or to the notifier provided by the constructor. (queue::priv::schedule_task): Do not schedule a nil task. Update comment. (queue::priv::schedule_tasks): Add a new member function. (queue::priv::do_bring_workers_down): Update comment. Protect access to "bring_workers_down" with tasks_todo_mutex to prevent a data race. Call pthread_cond_broadcast on the queue_cond unconditionaly to prevent some worker threads to keep waiting for ever. Also, protect the access to the queue_cond by the queue_cond_mutex to precent a data race. (queue::queue): Pass the notifier by reference. Update comment. (queue::schedule_task): Update comment. (queue::schedule_tasks): Define new member function. (queue::wait_for_workers_to_complete): Update comment. (queue::get_completed_tasks): Return a non-const vector. Update comment. (worker::wait_to_execute_a_task): Update several comments. Make the execution of the notification code to be synchronized (on the tasks_done_mutex). Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2017-02-24 09:58:16 +00:00
task_done_notify& notifier);
Use worker threads pattern to speed up some tests We are going to need to speed up more and more tests, and coding directly with libpthread for that can be tedious and bug-prone. So I devised an implementation for the worker threads design pattern instead, and used it to speed up some tests. * include/Makefile.am: Add the new abg-workers.h to source distribution. * include/abg-workers.h: New file. * src/Makefile.am: Add the new abg-worker.cc to source distribution. * src/abg-workers.cc: New file. * tests/test-utils.cc: Update copyright. Make get_src_dir() and get_build_dir() return a const char*, as opposed to returning a string. Make that const char reside in thread local storage, so that two concurrent threads can safely call these functions in parallel, without any race. * tests/test-utils.h: Make get_src_dir() and get_build_dir() return a const char*, as opposed to returning a string. * tests/test-abicompat.cc: Update copyright. Adjust for get_src_dir() and get_build_dir() change. * tests/test-abidiff.cc: Likewise. * tests/test-alt-dwarf-file.cc: Likewise. * tests/test-core-diff.cc: Likewise. * tests/test-diff-dwarf-abixml.cc: Likewise. * tests/test-diff-dwarf.cc: Likewise. * tests/test-diff-pkg.cc: Likewise. * tests/test-diff-suppr.cc: Likewise. * tests/test-lookup-syms.cc: Likewise. * tests/test-read-dwarf.cc: Likewise. * tests/test-read-write.cc: Likewise. * tests/test-types-stability.cc: Likewise. Use the new task queue type to run these tests in parallel. * tests/test-diff-filter.cc: Likewise. Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-01-15 12:34:16 +00:00
size_t get_size() const;
bool schedule_task(const task_sptr&);
Several fixes and enhancements to abigail::workers While making abipkgdiff to use the abigail::workers API to do away with using pthreads directory, it appeared that the abigail::workers API needs fixes and enhancements. Fixes ===== * Don't try to schedule a task if the pointer to the task is nil * Fix a data race when bringing workers (of a queue) down * Always try to wake up all waiting threads when bringing down queue workers. * Fix a data race when accessing the queue condition variable * Fix a data race when notifying listeners about the end of the job performed by the task. Enhancements ============ * Pass the "task done" notifier by reference, to the worker queue. Without this, the worker queue needs to copy the "task done" notifier by value. This implies that user code needs to provide task done notifier instances that come with potentially complicated copy constructors. By passing it by reference and by just re-using the notifier from the user code, we do away with the need for copying altogether. This also fixes some latent copying bugs. * Add a workers::queue::schedule_tasks() method This allows user code to schedule a vector of tasks at once. * make workers::queue::get_completed_tasks() return a non-const vector This enables user code to sort the completed tasks as they wish. * include/abg-workers.h (queue::tasks_type): New typedef. (queue::queue): Pass task_done_notify by reference. (queue::schedule_tasks): Declare new member function. (queue::get_completed_tasks): Return non-const vector. * src/abg-workers.cc (queue::priv::default_notify): New data member. (queue::priv::notify): Make this data member be a reference. (queue::priv::priv): Initialize the notify data member to either the new default_notify (if no notifier is provided by the constructor) or to the notifier provided by the constructor. (queue::priv::schedule_task): Do not schedule a nil task. Update comment. (queue::priv::schedule_tasks): Add a new member function. (queue::priv::do_bring_workers_down): Update comment. Protect access to "bring_workers_down" with tasks_todo_mutex to prevent a data race. Call pthread_cond_broadcast on the queue_cond unconditionaly to prevent some worker threads to keep waiting for ever. Also, protect the access to the queue_cond by the queue_cond_mutex to precent a data race. (queue::queue): Pass the notifier by reference. Update comment. (queue::schedule_task): Update comment. (queue::schedule_tasks): Define new member function. (queue::wait_for_workers_to_complete): Update comment. (queue::get_completed_tasks): Return a non-const vector. Update comment. (worker::wait_to_execute_a_task): Update several comments. Make the execution of the notification code to be synchronized (on the tasks_done_mutex). Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2017-02-24 09:58:16 +00:00
bool schedule_tasks(const tasks_type&);
Use worker threads pattern to speed up some tests We are going to need to speed up more and more tests, and coding directly with libpthread for that can be tedious and bug-prone. So I devised an implementation for the worker threads design pattern instead, and used it to speed up some tests. * include/Makefile.am: Add the new abg-workers.h to source distribution. * include/abg-workers.h: New file. * src/Makefile.am: Add the new abg-worker.cc to source distribution. * src/abg-workers.cc: New file. * tests/test-utils.cc: Update copyright. Make get_src_dir() and get_build_dir() return a const char*, as opposed to returning a string. Make that const char reside in thread local storage, so that two concurrent threads can safely call these functions in parallel, without any race. * tests/test-utils.h: Make get_src_dir() and get_build_dir() return a const char*, as opposed to returning a string. * tests/test-abicompat.cc: Update copyright. Adjust for get_src_dir() and get_build_dir() change. * tests/test-abidiff.cc: Likewise. * tests/test-alt-dwarf-file.cc: Likewise. * tests/test-core-diff.cc: Likewise. * tests/test-diff-dwarf-abixml.cc: Likewise. * tests/test-diff-dwarf.cc: Likewise. * tests/test-diff-pkg.cc: Likewise. * tests/test-diff-suppr.cc: Likewise. * tests/test-lookup-syms.cc: Likewise. * tests/test-read-dwarf.cc: Likewise. * tests/test-read-write.cc: Likewise. * tests/test-types-stability.cc: Likewise. Use the new task queue type to run these tests in parallel. * tests/test-diff-filter.cc: Likewise. Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-01-15 12:34:16 +00:00
void wait_for_workers_to_complete();
Several fixes and enhancements to abigail::workers While making abipkgdiff to use the abigail::workers API to do away with using pthreads directory, it appeared that the abigail::workers API needs fixes and enhancements. Fixes ===== * Don't try to schedule a task if the pointer to the task is nil * Fix a data race when bringing workers (of a queue) down * Always try to wake up all waiting threads when bringing down queue workers. * Fix a data race when accessing the queue condition variable * Fix a data race when notifying listeners about the end of the job performed by the task. Enhancements ============ * Pass the "task done" notifier by reference, to the worker queue. Without this, the worker queue needs to copy the "task done" notifier by value. This implies that user code needs to provide task done notifier instances that come with potentially complicated copy constructors. By passing it by reference and by just re-using the notifier from the user code, we do away with the need for copying altogether. This also fixes some latent copying bugs. * Add a workers::queue::schedule_tasks() method This allows user code to schedule a vector of tasks at once. * make workers::queue::get_completed_tasks() return a non-const vector This enables user code to sort the completed tasks as they wish. * include/abg-workers.h (queue::tasks_type): New typedef. (queue::queue): Pass task_done_notify by reference. (queue::schedule_tasks): Declare new member function. (queue::get_completed_tasks): Return non-const vector. * src/abg-workers.cc (queue::priv::default_notify): New data member. (queue::priv::notify): Make this data member be a reference. (queue::priv::priv): Initialize the notify data member to either the new default_notify (if no notifier is provided by the constructor) or to the notifier provided by the constructor. (queue::priv::schedule_task): Do not schedule a nil task. Update comment. (queue::priv::schedule_tasks): Add a new member function. (queue::priv::do_bring_workers_down): Update comment. Protect access to "bring_workers_down" with tasks_todo_mutex to prevent a data race. Call pthread_cond_broadcast on the queue_cond unconditionaly to prevent some worker threads to keep waiting for ever. Also, protect the access to the queue_cond by the queue_cond_mutex to precent a data race. (queue::queue): Pass the notifier by reference. Update comment. (queue::schedule_task): Update comment. (queue::schedule_tasks): Define new member function. (queue::wait_for_workers_to_complete): Update comment. (queue::get_completed_tasks): Return a non-const vector. Update comment. (worker::wait_to_execute_a_task): Update several comments. Make the execution of the notification code to be synchronized (on the tasks_done_mutex). Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2017-02-24 09:58:16 +00:00
tasks_type& get_completed_tasks() const;
Use worker threads pattern to speed up some tests We are going to need to speed up more and more tests, and coding directly with libpthread for that can be tedious and bug-prone. So I devised an implementation for the worker threads design pattern instead, and used it to speed up some tests. * include/Makefile.am: Add the new abg-workers.h to source distribution. * include/abg-workers.h: New file. * src/Makefile.am: Add the new abg-worker.cc to source distribution. * src/abg-workers.cc: New file. * tests/test-utils.cc: Update copyright. Make get_src_dir() and get_build_dir() return a const char*, as opposed to returning a string. Make that const char reside in thread local storage, so that two concurrent threads can safely call these functions in parallel, without any race. * tests/test-utils.h: Make get_src_dir() and get_build_dir() return a const char*, as opposed to returning a string. * tests/test-abicompat.cc: Update copyright. Adjust for get_src_dir() and get_build_dir() change. * tests/test-abidiff.cc: Likewise. * tests/test-alt-dwarf-file.cc: Likewise. * tests/test-core-diff.cc: Likewise. * tests/test-diff-dwarf-abixml.cc: Likewise. * tests/test-diff-dwarf.cc: Likewise. * tests/test-diff-pkg.cc: Likewise. * tests/test-diff-suppr.cc: Likewise. * tests/test-lookup-syms.cc: Likewise. * tests/test-read-dwarf.cc: Likewise. * tests/test-read-write.cc: Likewise. * tests/test-types-stability.cc: Likewise. Use the new task queue type to run these tests in parallel. * tests/test-diff-filter.cc: Likewise. Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-01-15 12:34:16 +00:00
~queue();
}; // end class queue
/// This functor is to notify listeners that a given task scheduled
/// for execution has been fully executed.
Use worker threads pattern to speed up some tests We are going to need to speed up more and more tests, and coding directly with libpthread for that can be tedious and bug-prone. So I devised an implementation for the worker threads design pattern instead, and used it to speed up some tests. * include/Makefile.am: Add the new abg-workers.h to source distribution. * include/abg-workers.h: New file. * src/Makefile.am: Add the new abg-worker.cc to source distribution. * src/abg-workers.cc: New file. * tests/test-utils.cc: Update copyright. Make get_src_dir() and get_build_dir() return a const char*, as opposed to returning a string. Make that const char reside in thread local storage, so that two concurrent threads can safely call these functions in parallel, without any race. * tests/test-utils.h: Make get_src_dir() and get_build_dir() return a const char*, as opposed to returning a string. * tests/test-abicompat.cc: Update copyright. Adjust for get_src_dir() and get_build_dir() change. * tests/test-abidiff.cc: Likewise. * tests/test-alt-dwarf-file.cc: Likewise. * tests/test-core-diff.cc: Likewise. * tests/test-diff-dwarf-abixml.cc: Likewise. * tests/test-diff-dwarf.cc: Likewise. * tests/test-diff-pkg.cc: Likewise. * tests/test-diff-suppr.cc: Likewise. * tests/test-lookup-syms.cc: Likewise. * tests/test-read-dwarf.cc: Likewise. * tests/test-read-write.cc: Likewise. * tests/test-types-stability.cc: Likewise. Use the new task queue type to run these tests in parallel. * tests/test-diff-filter.cc: Likewise. Signed-off-by: Dodji Seketeli <dodji@redhat.com>
2016-01-15 12:34:16 +00:00
struct queue::task_done_notify
{
virtual void
operator()(const task_sptr& task_done);
};
} // end namespace workers
} // end namespace abigail
#endif // __ABG_WORKERS_H__