etc/obs-StreamFX
0
0
Fork 0
mirror of https://github.com/Xaymar/obs-StreamFX synced 2024-11-14 07:45:06 +00:00
Code Issues Projects Releases Packages Wiki Activity
obs-StreamFX/source/util/util-threadpool.hpp

149 lines
3.8 KiB
C++
Raw Normal View History

util/threadpool: Optimize for dynamic thread pooling While the previous approach of a static thread pool worked, it was sub-optimal in its resource usage. Many of the threads would never see a single task, and simply permanently sleep. This seems like a good idea, except that sleeping threads still end up in the scheduler, and thus waste a tiny amount of resources. It is better to instead dynamically spawn threads when needed and only keeping the bare minimum around all the time. These dynamically spawned threads are also explicitly set to background priority which further reduces scheduling overhead. Finally optimizing the memory layout to prevent unwanted false sharing should also keep sporadic wake ups at a minimum. This new model should be able to handle many more tasks than ever before, but is still not as optimal as it could be.
2022-09-18 16:30:15 +00:00
// Copyright (C) 2020-2022 Michael Fabian Dirks
//
// This program is free software; you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation; either version 2 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with this program; if not, write to the Free Software
// Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA
util-threadpool: Implement global thread pool This thread pool can take large or small tasks and as such alleviates the burden of having a thread per source. Particularly for large setups, this drastically reduces the number of threads running in the background waiting for work.
2020-03-03 00:49:26 +00:00
#pragma once
cmake, code: Abuse pre-processor to disable/enable warnings
2022-08-29 10:29:44 +00:00
#include "warning-disable.hpp"
util-threadpool: Implement global thread pool This thread pool can take large or small tasks and as such alleviates the burden of having a thread per source. Particularly for large setups, this drastically reduces the number of threads running in the background waiting for work.
2020-03-03 00:49:26 +00:00
#include <atomic>
util/threadpool: Optimize for dynamic thread pooling While the previous approach of a static thread pool worked, it was sub-optimal in its resource usage. Many of the threads would never see a single task, and simply permanently sleep. This seems like a good idea, except that sleeping threads still end up in the scheduler, and thus waste a tiny amount of resources. It is better to instead dynamically spawn threads when needed and only keeping the bare minimum around all the time. These dynamically spawned threads are also explicitly set to background priority which further reduces scheduling overhead. Finally optimizing the memory layout to prevent unwanted false sharing should also keep sporadic wake ups at a minimum. This new model should be able to handle many more tasks than ever before, but is still not as optimal as it could be.
2022-09-18 16:30:15 +00:00
#include <chrono>
#include <cinttypes>
util-threadpool: Add missing includes
2020-04-01 12:32:25 +00:00
#include <condition_variable>
util/threadpool: Optimize for dynamic thread pooling While the previous approach of a static thread pool worked, it was sub-optimal in its resource usage. Many of the threads would never see a single task, and simply permanently sleep. This seems like a good idea, except that sleeping threads still end up in the scheduler, and thus waste a tiny amount of resources. It is better to instead dynamically spawn threads when needed and only keeping the bare minimum around all the time. These dynamically spawned threads are also explicitly set to background priority which further reduces scheduling overhead. Finally optimizing the memory layout to prevent unwanted false sharing should also keep sporadic wake ups at a minimum. This new model should be able to handle many more tasks than ever before, but is still not as optimal as it could be.
2022-09-18 16:30:15 +00:00
#include <cstddef>
util-threadpool: Implement global thread pool This thread pool can take large or small tasks and as such alleviates the burden of having a thread per source. Particularly for large setups, this drastically reduces the number of threads running in the background waiting for work.
2020-03-03 00:49:26 +00:00
#include <functional>
#include <list>
util-threadpool: Add missing includes
2020-04-01 12:32:25 +00:00
#include <memory>
util-threadpool: Implement global thread pool This thread pool can take large or small tasks and as such alleviates the burden of having a thread per source. Particularly for large setups, this drastically reduces the number of threads running in the background waiting for work.
2020-03-03 00:49:26 +00:00
#include <mutex>
util/threadpool: Optimize for dynamic thread pooling While the previous approach of a static thread pool worked, it was sub-optimal in its resource usage. Many of the threads would never see a single task, and simply permanently sleep. This seems like a good idea, except that sleeping threads still end up in the scheduler, and thus waste a tiny amount of resources. It is better to instead dynamically spawn threads when needed and only keeping the bare minimum around all the time. These dynamically spawned threads are also explicitly set to background priority which further reduces scheduling overhead. Finally optimizing the memory layout to prevent unwanted false sharing should also keep sporadic wake ups at a minimum. This new model should be able to handle many more tasks than ever before, but is still not as optimal as it could be.
2022-09-18 16:30:15 +00:00
#include <new>
common: Common header include for improved platform support This header includes all common data between headers used in the plugin. This should improve cross-platform compiling support whenever possible, as all platform-dependent common includes and defines can be done here.
2020-04-02 15:02:01 +00:00
#include <stdexcept>
util-threadpool: Implement global thread pool This thread pool can take large or small tasks and as such alleviates the burden of having a thread per source. Particularly for large setups, this drastically reduces the number of threads running in the background waiting for work.
2020-03-03 00:49:26 +00:00
#include <thread>
cmake, code: Abuse pre-processor to disable/enable warnings
2022-08-29 10:29:44 +00:00
#include "warning-enable.hpp"
util-threadpool: Implement global thread pool This thread pool can take large or small tasks and as such alleviates the burden of having a thread per source. Particularly for large setups, this drastically reduces the number of threads running in the background waiting for work.
2020-03-03 00:49:26 +00:00
util/threadpool: Optimize for dynamic thread pooling While the previous approach of a static thread pool worked, it was sub-optimal in its resource usage. Many of the threads would never see a single task, and simply permanently sleep. This seems like a good idea, except that sleeping threads still end up in the scheduler, and thus waste a tiny amount of resources. It is better to instead dynamically spawn threads when needed and only keeping the bare minimum around all the time. These dynamically spawned threads are also explicitly set to background priority which further reduces scheduling overhead. Finally optimizing the memory layout to prevent unwanted false sharing should also keep sporadic wake ups at a minimum. This new model should be able to handle many more tasks than ever before, but is still not as optimal as it could be.
2022-09-18 16:30:15 +00:00
namespace streamfx::util::threadpool {
typedef std::shared_ptr<void> task_data_t;
typedef std::function<void(task_data_t)> task_callback_t;
struct worker_info {
#if __cpp_lib_hardware_interference_size >= 201603
alignas(std::hardware_destructive_interference_size)
#endif
std::atomic<bool> stop;
#if __cpp_lib_hardware_interference_size >= 201603
alignas(std::hardware_destructive_interference_size)
#endif
std::mutex lifeline;
std::chrono::high_resolution_clock::time_point last_work_time;
std::thread thread;
};
class task {
task_callback_t _callback;
task_data_t _data;
std::mutex _lock;
#if __cpp_lib_hardware_interference_size >= 201603
alignas(std::hardware_destructive_interference_size)
#endif
std::condition_variable _status_changed;
#if __cpp_lib_hardware_interference_size >= 201603
alignas(std::hardware_destructive_interference_size)
#endif
std::atomic<bool> _cancelled;
#if __cpp_lib_hardware_interference_size >= 201603
alignas(std::hardware_destructive_interference_size)
#endif
std::atomic<bool> _completed;
#if __cpp_lib_hardware_interference_size >= 201603
alignas(std::hardware_destructive_interference_size)
#endif
std::atomic<bool> _failed;
util-threadpool: Implement global thread pool This thread pool can take large or small tasks and as such alleviates the burden of having a thread per source. Particularly for large setups, this drastically reduces the number of threads running in the background waiting for work.
2020-03-03 00:49:26 +00:00
util/threadpool: Formatting and allow cancelling of tasks
2020-05-10 00:50:52 +00:00
public:
util/threadpool: Optimize for dynamic thread pooling While the previous approach of a static thread pool worked, it was sub-optimal in its resource usage. Many of the threads would never see a single task, and simply permanently sleep. This seems like a good idea, except that sleeping threads still end up in the scheduler, and thus waste a tiny amount of resources. It is better to instead dynamically spawn threads when needed and only keeping the bare minimum around all the time. These dynamically spawned threads are also explicitly set to background priority which further reduces scheduling overhead. Finally optimizing the memory layout to prevent unwanted false sharing should also keep sporadic wake ups at a minimum. This new model should be able to handle many more tasks than ever before, but is still not as optimal as it could be.
2022-09-18 16:30:15 +00:00
task(task_callback_t callback, task_data_t data);
util/threadpool: Formatting and allow cancelling of tasks
2020-05-10 00:50:52 +00:00
util/threadpool: Optimize for dynamic thread pooling While the previous approach of a static thread pool worked, it was sub-optimal in its resource usage. Many of the threads would never see a single task, and simply permanently sleep. This seems like a good idea, except that sleeping threads still end up in the scheduler, and thus waste a tiny amount of resources. It is better to instead dynamically spawn threads when needed and only keeping the bare minimum around all the time. These dynamically spawned threads are also explicitly set to background priority which further reduces scheduling overhead. Finally optimizing the memory layout to prevent unwanted false sharing should also keep sporadic wake ups at a minimum. This new model should be able to handle many more tasks than ever before, but is still not as optimal as it could be.
2022-09-18 16:30:15 +00:00
public:
~task();
util/threadpool: Formatting and allow cancelling of tasks
2020-05-10 00:50:52 +00:00
util/threadpool: Optimize for dynamic thread pooling While the previous approach of a static thread pool worked, it was sub-optimal in its resource usage. Many of the threads would never see a single task, and simply permanently sleep. This seems like a good idea, except that sleeping threads still end up in the scheduler, and thus waste a tiny amount of resources. It is better to instead dynamically spawn threads when needed and only keeping the bare minimum around all the time. These dynamically spawned threads are also explicitly set to background priority which further reduces scheduling overhead. Finally optimizing the memory layout to prevent unwanted false sharing should also keep sporadic wake ups at a minimum. This new model should be able to handle many more tasks than ever before, but is still not as optimal as it could be.
2022-09-18 16:30:15 +00:00
public:
void run();
util/threadpool: Allow waiting for task completion
2021-10-25 19:56:49 +00:00
util/threadpool: Optimize for dynamic thread pooling While the previous approach of a static thread pool worked, it was sub-optimal in its resource usage. Many of the threads would never see a single task, and simply permanently sleep. This seems like a good idea, except that sleeping threads still end up in the scheduler, and thus waste a tiny amount of resources. It is better to instead dynamically spawn threads when needed and only keeping the bare minimum around all the time. These dynamically spawned threads are also explicitly set to background priority which further reduces scheduling overhead. Finally optimizing the memory layout to prevent unwanted false sharing should also keep sporadic wake ups at a minimum. This new model should be able to handle many more tasks than ever before, but is still not as optimal as it could be.
2022-09-18 16:30:15 +00:00
public:
void cancel();
util/threadpool: Formatting and allow cancelling of tasks
2020-05-10 00:50:52 +00:00
util/threadpool: Optimize for dynamic thread pooling While the previous approach of a static thread pool worked, it was sub-optimal in its resource usage. Many of the threads would never see a single task, and simply permanently sleep. This seems like a good idea, except that sleeping threads still end up in the scheduler, and thus waste a tiny amount of resources. It is better to instead dynamically spawn threads when needed and only keeping the bare minimum around all the time. These dynamically spawned threads are also explicitly set to background priority which further reduces scheduling overhead. Finally optimizing the memory layout to prevent unwanted false sharing should also keep sporadic wake ups at a minimum. This new model should be able to handle many more tasks than ever before, but is still not as optimal as it could be.
2022-09-18 16:30:15 +00:00
public:
bool is_cancelled();
public:
bool is_completed();
public:
bool has_failed();
public:
void wait();
public:
void await_completion();
};
class threadpool {
std::pair<size_t, size_t> _limits;
#if __cpp_lib_hardware_interference_size >= 201603
alignas(std::hardware_destructive_interference_size)
#endif
std::mutex _workers_lock;
std::list<std::shared_ptr<worker_info>> _workers;
#if __cpp_lib_hardware_interference_size >= 201603
alignas(std::hardware_destructive_interference_size)
#endif
std::atomic<size_t> _worker_count;
std::chrono::high_resolution_clock::time_point _last_worker_death;
#if __cpp_lib_hardware_interference_size >= 201603
alignas(std::hardware_destructive_interference_size)
#endif
std::mutex _tasks_lock;
#if __cpp_lib_hardware_interference_size >= 201603
alignas(std::hardware_destructive_interference_size)
#endif
std::condition_variable _tasks_cv;
std::list<std::shared_ptr<task>> _tasks;
util-threadpool: Implement global thread pool This thread pool can take large or small tasks and as such alleviates the burden of having a thread per source. Particularly for large setups, this drastically reduces the number of threads running in the background waiting for work.
2020-03-03 00:49:26 +00:00
public:
~threadpool();
util/threadpool: Optimize for dynamic thread pooling While the previous approach of a static thread pool worked, it was sub-optimal in its resource usage. Many of the threads would never see a single task, and simply permanently sleep. This seems like a good idea, except that sleeping threads still end up in the scheduler, and thus waste a tiny amount of resources. It is better to instead dynamically spawn threads when needed and only keeping the bare minimum around all the time. These dynamically spawned threads are also explicitly set to background priority which further reduces scheduling overhead. Finally optimizing the memory layout to prevent unwanted false sharing should also keep sporadic wake ups at a minimum. This new model should be able to handle many more tasks than ever before, but is still not as optimal as it could be.
2022-09-18 16:30:15 +00:00
public:
threadpool(size_t minimum = 2, size_t maximum = std::thread::hardware_concurrency());
public:
std::shared_ptr<task> push(task_callback_t callback, task_data_t data = nullptr);
public:
void pop(std::shared_ptr<task> task);
private:
void spawn(size_t count = 1);
util/threadpool: Formatting and allow cancelling of tasks
2020-05-10 00:50:52 +00:00
util/threadpool: Optimize for dynamic thread pooling While the previous approach of a static thread pool worked, it was sub-optimal in its resource usage. Many of the threads would never see a single task, and simply permanently sleep. This seems like a good idea, except that sleeping threads still end up in the scheduler, and thus waste a tiny amount of resources. It is better to instead dynamically spawn threads when needed and only keeping the bare minimum around all the time. These dynamically spawned threads are also explicitly set to background priority which further reduces scheduling overhead. Finally optimizing the memory layout to prevent unwanted false sharing should also keep sporadic wake ups at a minimum. This new model should be able to handle many more tasks than ever before, but is still not as optimal as it could be.
2022-09-18 16:30:15 +00:00
private:
bool die(std::shared_ptr<worker_info>);
util-threadpool: Implement global thread pool This thread pool can take large or small tasks and as such alleviates the burden of having a thread per source. Particularly for large setups, this drastically reduces the number of threads running in the background waiting for work.
2020-03-03 00:49:26 +00:00
private:
util/threadpool: Optimize for dynamic thread pooling While the previous approach of a static thread pool worked, it was sub-optimal in its resource usage. Many of the threads would never see a single task, and simply permanently sleep. This seems like a good idea, except that sleeping threads still end up in the scheduler, and thus waste a tiny amount of resources. It is better to instead dynamically spawn threads when needed and only keeping the bare minimum around all the time. These dynamically spawned threads are also explicitly set to background priority which further reduces scheduling overhead. Finally optimizing the memory layout to prevent unwanted false sharing should also keep sporadic wake ups at a minimum. This new model should be able to handle many more tasks than ever before, but is still not as optimal as it could be.
2022-09-18 16:30:15 +00:00
void work(std::shared_ptr<worker_info>);
util-threadpool: Implement global thread pool This thread pool can take large or small tasks and as such alleviates the burden of having a thread per source. Particularly for large setups, this drastically reduces the number of threads running in the background waiting for work.
2020-03-03 00:49:26 +00:00
};
util/threadpool: Optimize for dynamic thread pooling While the previous approach of a static thread pool worked, it was sub-optimal in its resource usage. Many of the threads would never see a single task, and simply permanently sleep. This seems like a good idea, except that sleeping threads still end up in the scheduler, and thus waste a tiny amount of resources. It is better to instead dynamically spawn threads when needed and only keeping the bare minimum around all the time. These dynamically spawned threads are also explicitly set to background priority which further reduces scheduling overhead. Finally optimizing the memory layout to prevent unwanted false sharing should also keep sporadic wake ups at a minimum. This new model should be able to handle many more tasks than ever before, but is still not as optimal as it could be.
2022-09-18 16:30:15 +00:00
} // namespace streamfx::util::threadpool
Reference in a new issue Copy permalink