etc/obs-StreamFX
0
0
Fork 0
mirror of https://github.com/Xaymar/obs-StreamFX synced 2024-12-18 06:20:14 +00:00
Code Issues Projects Releases Packages Wiki Activity

gs-vertexbuffer: Workaround for a foreign deallocation bug in libobs

Browse source 
The graphics subsystem in OBS is freeing memory it didn't allocate, resulting in stack/heap corruption and other kinds of messy situations. The official "fix" for this is to use bmalloc, but then you lose any kind of ability to re-use the same buffer for multiple vertex buffers or update on the fly without adjusting code that is possibly outside of your control (such as in libraries).

This works around the issues by "patching" the gs_vertexbuffer object to no longer hold a reference to the gs_vb_data object. Currently only D3D11 is supported for this kind of hack and it might break in a future obs-studio release.

PR fixing this Issue: https://github.com/jp9000/obs-studio/pull/993
This commit is contained in:
Michael Fabian 'Xaymar' Dirks 2017-11-03 00:13:07 +01:00
parent 3450c149aa
commit c7cfbff7dd
1 changed files with 18 additions and 4 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

22
source/gs-vertexbuffer.cpp
View file

@ -20,10 +20,10 @@
#include "gs-vertexbuffer.h" #include "gs-vertexbuffer.h"
#include <stdexcept> #include <stdexcept>
extern "C" { extern "C" {
#pragma warning( push ) #pragma warning( push )
#pragma warning( disable: 4201 ) #pragma warning( disable: 4201 )
#include <libobs/obs.h> #include <libobs/obs.h>
#pragma warning( pop ) #pragma warning( pop )
} }
const uint32_t defaultMaximumVertices = 65535u; const uint32_t defaultMaximumVertices = 65535u;
@ -78,6 +78,20 @@ GS::VertexBuffer::VertexBuffer(VertexBuffer& other) : VertexBuffer(other.m_maxim
GS::VertexBuffer::~VertexBuffer() { GS::VertexBuffer::~VertexBuffer() {
if (m_vertexbuffer) { if (m_vertexbuffer) {
std::memset(&m_vertexbufferdata, 0, sizeof(m_vertexbufferdata)); std::memset(&m_vertexbufferdata, 0, sizeof(m_vertexbufferdata));
// Workaround API stupidity in obs-studio.
// See PR: https://github.com/jp9000/obs-studio/pull/993
if (gs_get_device_type() == GS_DEVICE_DIRECT3D_11) {
#ifdef _DEBUG
*(reinterpret_cast<intptr_t*>(m_vertexbuffer) + 14) = 0;
#else
*(reinterpret_cast<intptr_t*>(m_vertexbuffer) + 13) = 0;
#endif
} else {
//*(reinterpret_cast<intptr_t*>(m_vertexbuffer)+14) = 0;
}
obs_enter_graphics(); obs_enter_graphics();
gs_vertexbuffer_destroy(m_vertexbuffer); gs_vertexbuffer_destroy(m_vertexbuffer);
obs_leave_graphics(); obs_leave_graphics();