While Gaussian Blur is not a Blur type that really benefits much from linear sampling, it can be used for a slight quality and gpu usage reduction. However for Area and Directional Blur there is a better alternative: Dual Filtering Blur. And as with all other currently implement Linear versions of Blur, only Area and Directional Blur are supported.
This type of Gaussian Blur also has the loading hitch that exists in normal Gaussian Blur.
Related: #45, #6
Gaussian Blur is another Blur that now supports the new system, increasing the maxium Blur size to 128 and adding support for Rotational and Zoom blur. Various optimizations were done to the actual shader code which further reduced the GPU usage.
Currently the Gaussian curve is recalculated when the blur is first created, which can lead to a short hitch due to it having to search for the correct kernels. This is currently unavoidable and expected behavior until a better solution is found.
Related: #45, #6
Box Blur is a prime candidate for Linear optimizations, and as such reducing the total necessary samples by about half. However due to the reduction in samples, only Area and Directional Blur are supported.
Related: #45, #6
Box Blur is the first to be on the new system and has received all the new features and optimizations available. The maximum Blur size has been increased to 128, Rotational and Zoom Blur are supported and a small optimization has been done to the shader.
Related: #45, #6
This is the new code base for blur effects, which is much more effective at being re-usable than the old code base. It allows for much better extendable behavior and uses an interface and factory pattern, instead of hardcoding supported features.
Small performance optimization as this removes the extra texture render that would be done by libobs which we don't actually need as we already have a texture to render to for it.
Rendering happens with cached Geometry that is only updated when necessary, such as when the user changes settings. This optimization was necessary to reduce CPU and GPU usage as we can simply re-use the same geomtry instead of recalculating it.
However, the actual source size was never checked. That meant that when a source changed size through any means, the filter would not update the geomtry accordingly, and the result was a squished or stretched image.
With this, the source size is now checked each tick.
Fixes#48
The number of files in the source directory was a little bit much and just made file naming more complex than it had to be. Therefore all files were moved into subdirectories where it matters.
Filters now reside in source/filters/, Sources in source/sources/, OBS Wrappers in obs/, OBS GS Wrappers in obs/gs/, Transitions will reside in source/transitions, Graphics Helpers will be in gfx/.
MipGenerator.Strength was renamed to MipGenerator.Intensity, which better fits the actual intention of the property. Additionally all the mipmap generator options are now correctly named.
Gaussian blur used to have a hard edge which was caused by the filter width being wider than the actual sampling range, which is what is determined by the size option. In order to fix this, we have to figure out the proper filter width for a sampling range, so that we avoid any and all hard edges.
Additionally removes the old texture method entirely.
Fixes#44
Source Mirror would incorrectly crash if an Audio only source was selected, as it expected a video frame to be present of any size. This fixes the problem by first testing if there is any video information to render and then just catching any exception that happens during rendering.
The normal OBS behavior is to cache files that are loaded from disk, however this is not the behavior we want at all, as it can miss file updates as it never checks the disk for an updated file.
So instead of relying on OBS, this implements the file reading and effect creation without the cache. If a cache is necessary, it would not be the job of this class.
This allows increasing or decreasing the accuracy of the Signed Distance Field at runtime without requiring the initial source to be resized. Use case for this would be small text where the higher quality would only be noticable on the shadow and not the text itself.
Sources were rendering with a black border around them if they had a soft fade, which is due to how the shadow calculated the image sample. With the new shader code this is now fixed and the source looks like it should be. Additionally this removes the bug where enabling any shadow would cause only the texels to draw that were above the threshold.
Additionally this adds support for inverse gradients (min > max) and negative gradients for outer shadows instead of only positive gradients. This technically allows for cleaner shadows.
The nature of this Signed Distance Field based Filter is not to only draw shadows, but to also draw an outline or even glow around a source. Therefore the name "Inner/Outer Shadow (SDF)" does not fit well with the intent of the actual Filter.
Allows for enumerating any kind of source, as long as it has a name and is not private. As obs_enum_scenes is a 23.x and above function, this class should be used instead to remain backwards compatible.
Fixes the m_source_texture = nullptr bug, which happened due to m_tick and m_last_tick both being 0, as no video_tick is called when the source is first created. Instead a boolean flag is now used that is reset to false on each video_tick.
This replaces the use of gs_texture_t* with gs::texture, further improving stability. Additionally some of the useless checks were replaced or reduced.
The filters are always the same for every language and should not be translated, instead only the file type should be translated. This way bad translations will not affect the options that the user can select.
The Transform filter will now no longer render the child source more than once per tick(), resulting in an overall speed up for heavy sources. This also applies to mipmapping and shape rendering. Any other calls to video_render will instead just use the cached texture.
Additionally the crash on exit has been fixed which was caused by strings.hpp containing static const char*s and using these directly in obs calls. Instead we now use #define for those property names
Rendering will now use a cached version of the mirrored source in order to reduce rendering cost drastically for heavy sources or scenes. Additionally rescaling now uses the internal scene instead of being a custom implementation, allowing for some new things.
Rescaling now has a new option: Bounds Type! This option allows control over just how a source is scaled, by default it is set to Stretch, but there are other options that can keep the aspect ratio of the mirrored source in tact.
Additionally the ScalingMethod struct has been replaced with obs_scale_type, which means that Bilinear Low Resolution is now no longer an option.
Implements #35
Initialization happens in the constructor again, as using a delayed initialization only caused update() to be slower.
Additionally refactored some variable names to make more sense.
load() is guaranteed to happen after all sources have been created and no earlier, and we can use update() to guarantee that we don't need to duplicate our initialization code in case we go back to initializing earlier than load().
This fixes the multi-reference issue that caused Source Mirror to keep a permanent reference to a source that should have been released.
The obs::audio_capture was also updated to now use the util::event code for improved callback support. There are a bunch of new additional checks in the acquire_input function that prevent bad references from happening, and is also now self-cleaning through the use of obs::source
By using util::event a single obs::audio_capture can now have multiple callbacks for the same data, without requiring additional obs::audio_capture to be present. Additionally it allows us to stop listening on an obs callback if there are no actual listeners, and start listening as soon as there is a listener.
Move and Copy constructors are currently not supported by this class and thus have been marked deleted. Internally also now using obs::source to keep track of the used source.
There was no proper implementation for all copy and move constructors and operators before, which meant that occasionally the default behavior took over. This is not what we want, as the default behavior doesn't deal well with reference counted pointers.
Related: #32, #33
While Linux was not an original goal of the project, it should still be supported out of the box. Therefore a number of changes are contained in this changeset:
- All C++ .h files were renamed to .hpp.
- All C includes (<....h>) were replaced with C++ includes (<c...>) and missing includes were added.
- std::memset and std::memcpy was replaced with memset and memcpy.
- All anonymous structs were removed where necessary.
- `extern "C"` was removed where it wasn't needed.
- #pragma warning was placed behind an #ifdef _MSC_VER
- The macros for `min`, `max` and `clamp` were removed as they were not used.
- `-fpedantic` was added to the GCC flags for bitmask support.
- `gs::rendertarget_op` is now declared before use.
- std::exception(const char*) was replaced with std::runtime_error(const char*).
- Added aligned_alloc and aligned_free macros for GCC and MSVC support.
- Replaced use of `sprintf_s` with `snprintf`.
- Placed additional guards around windows only code.
Additonally some changes were made that do not affect Linux:
- `NOMINMAX` and `NOINOUT` were removed.
Fixes: #27Fixes: #13
Directional Blur (also known as Motion Blur in some art software) enables full control over the direction the passes are applied in, which can further be combined with the scaling option. With this very simple blur effects can have a massively bigger influence.
Step Scaling allows for improved control over individual Blur passes and can be combined with Directional Blur to imitate certain effects. It also allows smaller Blur sizes to cover a larger area, although with a drastic loss in final quality. For example, if someone is streaming at 720p and wants to have their dynamic 2160p background blurry to a very high degree, this option would allow that at next to no quality loss in the final stream. The actual values that need to be used vary for each source and encoder however, so they will have to be chosen carefully.
Translucent sources previously rendered with wrong alpha, which caused sources to look different than they should. With the new code, this is no longer the case and instead now renders the source correctly.
Additionally the modernized code should reduce any problems with rendering by reducing the amount of duplicated manually written code and instead using classes.
This speeds up Gaussian Blur and Linear Gaussian Blur drastically reduces time spent reading textures and instead uses existing registers - maximizing time spent reading the actual image texture.
See Also: #21 Blur Quality
Similar to Linear Box Blur, this version of Gaussian Blur reduces the total number of sample by up to n. This results in a total sample count (per pass) of O(n+1) for even radii and O(n+2) for odd radii. The quality sacrificed to do this is higher this time, though careful adjustment of the halfTexelDelta value can bring it much closer to normal Gaussian Blur. The current offset however had no noticable effects on visual quality.
See Also: #21 Blur Quality
Linear Box Blur abuses the fact that with Linear Sampling we can sample up to four adjacent texels at the same time and get a correct result for Box Blur back. Using this the total number of sample for Box Blur is reduced by n, making the total either n+1 (Even Radius) or n (Odd Radius).
Additionally all blur effect files have been merged into a single blur.effect file to reduce the time required to change a single parameter name. New blur effects should be added as a new technique instead of as a new effect file.
See Also: #21 Blur Quality
Adds Inner/Outer Shadows for dynamic sources based on signed distance field generation. This is fast, but does add a bit of latency when it comes to updates - which means that moving objects will leave a trail before the generator has a chance to update.
Fixes#3
Due to the lack of checking aud and aoi, a rare race condition crash can be observed with audio mirroring enabled on closing OBS. In this case, aoi is used even though it is null, causing OBS to crash instead of closing normally.
Fixes#22
Adds full support for Scenes as mask input so that you can re-use your overlay scene as a blur input and have the blur follow behind your overlay.
Closes#14
* Removes the old 'Region' fields and places them under a 'Mask' option that can do much more.
* Supported Mask types: Region, Image, Source.
* Image and Source mask types allow for a color filter and multiplier.
* Now using a Factory and Instance approach to simplify and beautify the code at the same time.
* Additionally we now keep track of created filters and only load data if there is a filter active, and unload data if there are no more filters. This reduces memory and GPU overhead when no filter is active, but adds a first time filter creation overhead.
* Variables and functions now use snake_case.
The filter will now automatically rescale the parent stack into the next best power of two size. With this, even non-power-of-two sources can now be mipmapped semi-correctly.
To accurately support mipmapping even for npot textures, this feature would have to be built into OBS and OBS would have to stop refusing to create textures with mipmaps that are not a power of two in size. Almost all common Direct3D 11 (except Intel) are capable of npot mipmaps at full speed, while OpenGL usually depends on the GPU and Driver used.
The '3D Transform' filter now support mipmapping using the Linear generator. This results in smoother images when the shape is squished or distant, instead of a pixelated mess.
This enables full mipmapping support for textures with a shader view that allows accessing different mip levels. In order to access mip levels, you have to specify gs::texture:🎏:BuildMipMaps when creating the texture, as OBS currently forces the maximum mip level to 1 even if you actually have mip data available.
This class is an attempt at adding dynamic mipmapping support to OBS, which is lacking this feature. It is pretty much a hack until I figure out how to do it for both d3d11 and opengl and can make a PR for obs-studio to include this ability by default.
The Custom Shader Filter is now capable of rendering a custom shader using the gfx::effect_source class as a backend. An example shader is provided for starting off, more advanced examples may come later.
Related: #5
Rendering is now possible, although some parameter types are not yet supported. So far, booleans and floats will work fine, integers will cause an error in OBS Studios rendering code for an unknown reason.
The Blur Filter can now be applied to a region inside the source itself, the inverse of that region, and/or a feathered version of that region. This allows for easier scene setups where only some parts need to be blurred, but the rest can be left as is.
Fixes#12
From this point on, Source Mirror is now capable of real-time mirroring of Video and Audio. This can help if you need different filters per scene for your microphone or voice chat, depending on the scene (audio ducking for pause scene, no audio ducking for live gaming).
gfx::effect_source is the base class for all Custom Shader Sources and Transitions, which reduces the overall workload to a single file, but unfortunately also reducing the effective customization per source a bit.
Converts a String to an std::pair of int64_t (long long), which contain the size or 0 if none could be parsed. Any delimiter except digits(0-9), a minus sign(-) or a plus sign(+) between width and height are allowed. If a plus or minus sign is used as a delimiter, it must immediately be followed by the second size number. This allows for formats such as: 100x100, 100:100, 100p100, 100@100, 100+100 and so on, but not formats such as 100+:100, 100ThisIsSomeReall+yLongText100, etc.
The parameter 'allowSquare' also determines what to do when the height parameter is not found. A value of true will have the function return <width,width> instead of <width,0>.
This utility class is used to quickly render and retrieve a Texture from a source. It should be used in place of manual rendering since it can be updated quickly, fixing outstanding issue in all places of the plugin instead of just one.
The custom allocator occasionally returned memory that was aligned, but did not have enough space to store the actual size due to a calculation error in the size. This resulted in situations where allocating 1022 bytes would give you a writable buffer of only 1020 bytes or less, or also known as writing into unknown memory, possibly even the heap. This is now fixed by doubling the padding used.
Additionally it will now default to using standard allocators, which should work better and rely on the Compiler.
- Shaders now have all promised special variables available to them.
- 'Pass' (int) is no longer a special variable.
- Files should now properly update every half second.
- Fixed a ton of compiler warnings.
- Forced loading shaders from text instead of files to avoid caching.
- Fixed file modification tests always failing.
- Actually renders shaders now.
Known issues:
- Parameters aren't updated with file modifications unless the file is actually changed in the UI.
HasParameter can be used to safely check if a parameter exists and such should be preferred over try-catch blocks. Additionally, SetFloat2, SetFloat3, SetFloat4, SetFloatArray, SetInteger2, SetInteger3, SetInteger4 and SetIntegerArray should no longer cause rendering issues due to invalid buffer sizes.
Custom Shader allows you to write your own effect files and just have them applied to your source(s). It will dynamically update the properties to match the parameters in the source as well as offer some special parameters to the shader.
# Conflicts:
# CMakeLists.txt
# data/locale/en-US.ini
# Conflicts:
# data/locale/en-US.ini
Scaling was previously incorrectly rendering the source with another effect forced onto it, resulting in slower rendering and some sources that would no longer render properly.
Additionally the new option allows the user to have the source render at the original resolution in order to allow previously applied transform to stay identical. The rescaling however will no longer apply to filters after this source then, thus the speed bonus is lost.
Also categorized the localization file and adds descriptions for existing and new properties for Source Mirror.
Allows code to directly use GS::Vertex for their own purposes without having to rely on GS::VertexBuffer, which comes with the added overhead of allocating GPU memory.
Changes the GS::VertexBuffer storage to be one continuous buffer that is properly aligned and is also now used for GS::Vertex. This halves the necessary memory, removes reallocation cost and removes the copy necessary to get things onto the GPU.
Related: #9
This unifies the logic in GS::IndexBuffer and GS::VertexBuffer so that both can take the same amount of vertices. Additionally the limit for vertices was increased to 16777216 from 65536 to allow for proper models to be stored.
The previous fix unfortunately didn't actually fix it and instead made the crash invisible at first and then corrupt the heap at a later point. With this, VS2013 and VS2015 create code identical to what VS2017 creates, and no longer seem to crash.
Related: #9
This fixes the crash on creation issues, but a crash on exit still happens with the plugin installed. Unsure what exactly is causing it but it looks like something is writing into heap memory.
Related: #9
A bug in Visual C++ 2013 32-bit & 2015 32-bit causes the C++ compiler to incorrectly align the vec3 and vec4 structs to 8-byte instead of 16-byte, resulting in a crash if the target PC supports SSE. Visual Studio 2017 and 64-bit builds are not affected.
Related: #9
The source allows you to apply effects to the same source without requiring a new instance of the source. Any changes done to the original source also apply to the mirror, so there is less total work that needs to be done.
It can also rescale the source, allowing you to use the same source as a cheap instant backdrop with Blur for example. What you do in the end is completely unwritten and up to you to decide.
An earlier commit changed how Blur filter rendered and thus m_technique was no longer being initialized, resulting in the check randomly failing if the memory was just right.
This also adds some more extra logging to the instance creation in case things fail and fixed one MSVC warning.
Fixes#1
- Fixed Gaussian Blur having color issues with radii that aren't a power of 2.
- Improved error reporting to better debug problems where Blur doesn't render.
- Switched from manual object management to managed classes.
- Lots of code cleanup for maintainability.
The graphics subsystem in OBS is freeing memory it didn't allocate, resulting in stack/heap corruption and other kinds of messy situations. The official "fix" for this is to use bmalloc, but then you lose any kind of ability to re-use the same buffer for multiple vertex buffers or update on the fly without adjusting code that is possibly outside of your control (such as in libraries).
This works around the issues by "patching" the gs_vertexbuffer object to no longer hold a reference to the gs_vb_data object. Currently only D3D11 is supported for this kind of hack and it might break in a future obs-studio release.
PR fixing this Issue: https://github.com/jp9000/obs-studio/pull/993
End goal of this is to be an anisotropic filtering purely on the GPU without any actual mipmaps in the graphics subsystem. The basic idea behind it is that we can create all possible mipmaps in a twice as big texture as the original and then can efficiently render it no matter the GPU features as it will only consume one sampler.
The texture that it outputs will have both horizontal, vertical and full mipmaps in it. Horizontal mipmaps reduce Width, Vertical mipmaps reduce Height and full mipmaps reduce both. This means that LoD levels are both possible in the X and Y direction, allowing for much greater precision mipmapping on all GPUs.
This is a temporary solution until we can directly copy updated textures to a mipmap level.
This option allows applying the blur to other color formats such as YUV. Bilateral Blur benefits the most from this, resulting in smoother images at lower kernel sizes.
* Added a 'Bilateral' de-noising blur, currently very resource hungry. It also has two extra sliders called 'Smoothing' and 'Sharpness', the former controls how smooth the output is
* Updated effect loading to be global instead of per instance.
* Changed default blur size to 5px.
* Simplified render loop (see previous effect commit).
This is not very optimized at the moment and will take quite a bit of GPU time. Will have to spend some time later to optimize it and perhaps add directional blur ("Motion Blur" to it).
* Fixed the perspective mesh not filling the entire frame with the default settings.
* Changed the maximum range of 'Field of View' to 179 degrees instead of 180 degrees.
* Changed the direction the perspective camera looks at to +Z instead of -Z. This should feel more natural for users of other 3D programs.
* Changed the 'Position (Z)' default to 0.
* Added translation strings for 'Field Of View', 'Position', 'Scale' and 'Rotation'.
FOV for the perspective camera was previously set to 180° (or more) resulting in really small images. Also added proper rotation support with options to choose which axis to rotate first.
