Both Rec. 601 and sRGB looked extremely wrong before, resulting in weird or warped colors. Since it is very difficult to find up to date and accurate information on standards, we should simply go for what has the most widespread support.
There is hardly any reason for us to recalculate everything all the time. LUTs can cache the work once, and then re-use it every time necessary, drastically reducing the impact of Color Grading by almost 60% (on some GPUs even more). Additionally this fixes the negative gamma issue, which plagued the filter for a while.
In the future, once PR 4199 (https://github.com/obsproject/obs-studio/pull/4199) has been merged, we can cut away one intermediate rendering step currently required to make the effect work. Hopefully this will be with the 27.x release of OBS Studio.
For simple image and video editing, LUTs (Look-Up Tables) are vastly superior to running the entire editing operation on each pixel - especially if all the processing can be done inside a single shader.
Due to the post-processing requirements for our LUTs, we are limited to 8 bits per channel - though clever use of the unused Alpha channel may result in additional space. For our purposes however, this is definitely enough.
A complete redesign of the component and dependency system is necessary in order to support additional platforms, such as MacOS and other Linux platforms. Additionally it results in a much cleaner code base, which is less confusing overall.
Eventually it might be necessary to push components of StreamFX into their own CMake projects, as it is getting kind of complex now. Especially with the push for a proper plugin manager, things get dicey for big plugins like StreamFX.
Ignoring the data parameter during initialization results in duplication and some third party plugins not working as expected, so it's better to not ignore it.
Fixes#315
Adds support for the AMD Advanced Media Framework H.264 and H.265 encoders via FFmpeg. The majority of settings are supported, and the UI/UX experience mimics that of the NVENC implementation. Various settings are left out due to their complexity and should be controlled via the custom parameters field.
Using the obs_module_file and obs_module_config_path macros works okay, but it comes with a slight overhead as well as additional requirements when passing it to C++ functions that expect certain rules to be fulfilled. By instead wrapping the actual functionality into our own functions and using those we can avoid most of the issues that come with the old approach.
Related: #359
This fixes a bug with older Qt versions which would not remove the modal window from the parent in the setModal() call, resulting in an unusable parent window.
Fixes#368
Improves the previous logic and makes it compatible with the new additions in 26.x, such as sRGB. This was previously broken as the focus was on existing features which could be tested without requiring a compiler to be installed.
Incorrect understanding of how sRGB works with RGB and YCC/YUV formats also caused sRGB to be treated as RGB when I444 was selected. This should also now be fixed, hopefully permanently.
Fixes#331
In order to work around a bug in the OBS Studio UI code, we have to swap our modal status right before showing and right after hiding, so that the OBS Studio tray menu continuous working correctly. This is a bit of a weird solution, but it does work as expected.
Related: https://github.com/obsproject/obs-studio/issues/3518
Implements a manual and automatic update checker with support for both release and testing update channels, allowing users to stay as up to date as possible. It is fully compliant with privacy regulations around the world, as it stays completely silent and inactive until the user gives the Ok to connect to GitHub for the latest releases.
Removes the NVENC entry on systems without an NVIDIA GPU by checking if the library for it can be loaded. If it can't be loaded, it's likely that the user does not have a system with NVENC capabilities - and guaranteed that they can't use the encoder as FFmpeg relies on these libraries.
Adds a utility class and functions to load libraries and symbols from libraries somewhat safely. Libraries are immediately unloaded when the last reference to them is lost, so the shared_ptr should be stored if the library is actually needed.
Changes applied:
* Moved utility files to /util/.
* Removed unused #includes.
* Removed unused ::ffmpeg::tools function.
* Removed unused variables.
* Fixed missing parentheses in the version macro.
* Fixed missing override on virtual function overrides and removed unnecessary virtual keyword from them.
* Disabled additional warning for ATL headers on MSVC only.
* Replaced direct printf parameters with their macro equivalent.
* Replaced C-style casts with C++-style casts.
* Applied clang-format again after an earlier change to the CMake file broke the integration for it.
The levels 5.2 and above were added in a newer FFmpeg version than OBS Studio is shipping with. Ideally we'd take these options directly from FFmpeg, but that is for a future commit.
Fixes the issue where "high444p" shows as "high", and "high" shows as "main", caused by the internal option mapping never being correct across version changes. Instead of keeping our own copy of things that has to be adjusted for each FFmpeg version, it's easier to just use the FFmpeg values and names.
* Changes the encoder name to 'streamfx-{name}' from 'streamfx--{name}' as the latter is a typo, but adds a proxy to still support the latter in bad configurations.
* Some of the warning messages have been improved in order to better support end-users, and support for the new encoder error messages has been added.
* Adds support for the is_hw argument instead of blindly relying on obs_encoder_get_caps() which actually returns the wrong values due to rerouting.
* Fixed handler-less encoders showing up in the UI outside of debug builds.
Due to an oversight in the code, the fallback code would simply fail to work as the encoder was always the zero-copy version. This resulted in weird crashes and encoders not working that used to work before.
Fixes#295
* "Quality" Minimum/Maximum is actually QP Minimum/Maximum
* Bitrate Limits is now just Limits
* Buffer Size and Quality Target have been moved into "Limits".
Proxies allow for older configurations to work fine on newer versions, without having to manually adjust the scene collection to match the new ids at all. Thanks to the migration system we can freely support any number of old versions, as long as we write migration code.
Occasionally, mostly due to other sources rebuilding their UI, an out-of-order lock freeze can be observed with Source Mirror. This is unwanted, so we need to move the freezing logic into a place where freezing shouldn't happen.
Fixes#228
Actually fixes#61
Moves the menu for StreamFX to the primary menu, so that it is not hidden under tools. This makes it clearer to the user if their installation of StreamFX is working correctly, in addition to reducing the steps necessary to use the menu.
Also the 'About StreamFX' dialog now actually shows up for every update, as expected.
Adds support for enumerations, a different way of selecting how something should behave in a shader. Enumerations rely on a continuous list of values, and will automatically detect how many values there are in the enumeration. Only non-vector types are supported as enumeration entries, and array/vector parameters can have each member set to a different enumeration value.
Furthermore suffixes now are properly assigned, and 'bool' no longer causes shaders to stop rendering. Additionally by inlining some functions and using std::string_view we can achieve a slightly better performance than before.
Using std::string_view over std::string (and const std::string&) has the advantage that we skip potential temporary std::string objects that are immediately thrown away, thus slowing down the code. It can also be implicitly cast to std::string, which makes it compatible with existing code that uses std::string.
The high priority CUDA stream causes libOBS to be at a lower priority than the tracking, which is not what we want. Instead we want tracking to be incomplete in those cases, rather than slowing down encoding and other things.
Geometry updates are also now done once per frame instead of one per tracking update, which should improve the smoothness without affecting performance too much. Additionally all tracking info is now in the 0..1 range, which drastically simplifies some math - especially with texture coordinates.
To deal with tracking and updates being asynchronous, a very simple approximation of movement velocity has been added. This is mostly wrong, but it can bridge the gap where tracking updates are slower, as the values are all filtered anyway.
Adds a new CMake option "ENABLE_PROFILING" which enables all CPU and GPU performance profiling available in StreamFX for tracking what's actually causing things to be slow.
Asynchronous rendering allows the GPU to perform work while the CPU performs other work, and is significantly faster than lockstep immediate rendering. By reusing existing render targets we can see a performance improvement of up to 500%, while still doing the same things.
The new libOBS API allows us to directly access the underlying API instead of having to mess around in memory. By using it we can avoid crashing in case the compiler for it is different, or in case the actual back end structure changes.
Additionally the mostly unimplemented and unused options have also been removed, which streamlines the use of this class even further and reduces both shader and code complexity.
Finally by optimizing the use of the internal render target we can achieve a speed up of up to 3000% over the old way, allowing for many more mipmapped filters.
Q_INIT_RESOURCE and Q_CLEANUP_RESOURCE can't be called from within a namespace and instead have to be in outside of the namespace, so by moving them into small inline functions we can fulfill this restriction.
Related: #192#155
Fixes rendering at unexpected sizes by first rendering to a render target and then rendering the contents of that render target to the frame buffer instead. This also prevent rendering twice or more, which might cause severe FPS impact.
Implements support for various new UI features that weren't possible up until now, such as an 'About StreamFX' window with a thank you to everyone that supported the project up until now.
Through converting the code to a threaded asynchronous approach, the libOBS video renderer no longer has to wait on our tracking code to run, and we can enjoy a little bit of extra calculation time before we actually have to do anything.
However due to the remaining synchronization with the Direct3D11/OpenGL context, it is not entirely safe to spend a full frame tracking as libOBS will then start skipped/dropping frames. Even though the priority of the stream is now increased, this still means that we can't just sit around and have to quickly finish all work.
Related #150
Load additional functions from CUDA and add new enumerations to support them:
* cuDevicePrimaryCtxSetFlags allows us to sched scheduling mode for the GPU.
* cuCtxgetStreamPriorityRange allows us to check which priority levels are supported.
* cuStreamCreateWithPriority allows us to create streams with non-default priority.
The scheduler mode is now set to yield so that other threads can do work when we hit an eventual stalling problem. Streams can also now be created with higher priority and different flags, if necessary. In most cases this should allow CUDA resources to execute even while the GPU is under heavy load.
Previously sources had to manually implement migration code, which resulted in unresolvable regression issues due to the lack of version and commit tagging. With the new migration code, all sources automatically have this version and commit tagging at all times, and as such can now have a temporary regression fixed without the user needing to change any values manually.
As OBS Studio locks some mutexes in a different order depending on what actions are being done, using modified_properties for GPU work causes things to freeze in place. Instead have users manually click the refresh button when they changed files in order to prevent this freeze from happening.
Fixes: #118
With this, GCC 8 and above should now be able to compile the project both in obs-studio and as a standalone install. Some features are currently still not fully supported and require extra work, but the majority of things are supported and work out of the box. Exact feature parity can be looked up here on the wiki: https://github.com/Xaymar/obs-StreamFX/wiki/Platform-Feature-Parity
Related: #119#98#30
This header includes all common data between headers used in the plugin. This should improve cross-platform compiling support whenever possible, as all platform-dependent common includes and defines can be done here.
Ever wished you had a professional camera operator to highlight and follow the action, ensuring the audience never misses a beat? Thanks to NVIDIA, you can now do this at home for free! The new NVIDIA AR SDK unlocks augmented reality features, including motion tracking for faces.
This allows me to provide you with an automated zoom and cropping solution for your video camera to transform your streams into a slick, polished broadcast, where you’ll always be the star of the show. Don’t forget - everything is customizable so the possibilities are endless. You can even recreate that Futurama squinting meme if you wanted to (with some scripting)!
The filter requires compatible Nvidia RTX hardware and the Nvidia AR SDK Runtime to be installed ahead of time. This filter is considered "stable" and shouldn't change much from version to version.
Due to the 'nvcuda' library being part of the driver, it falls in a clause of the GPL which allows us to load and interface with system drivers. Since we can't rely on Nvidias headers here (incompatible license), most of this was pulled from FFmpeg and other things were found out via testing.
'Time.x' gets inaccurate if OBS Studio is running for more than two hours, therefore we have to do something to fix it. By allowing the shader code to control when things loop using 'Time.y' (0..1) and 'Time.z' (the number of times 'Time.y' wrapped back to 0), a much more stable animation can be achieved.
These allow you to apply any kind of filtering to a any source, using just standard HLSL. Just like transitions, one extra parameter is set called 'InputA'.
Fixes#95
With this, the first proper shader effect is now possible. By using the four new automated shader parameters 'InputA', 'InputB', 'TransitionTime' and 'TransitionSize' you can write your own transition in HLSL.
Fixes#96
Due to render logic required for transitions, some of the render logic is split into an additional function called 'prepare_render'. Additionally the storage for some temporary objects has been removed as it these objects usually do not outlive their rendering time anyway.
Related: #96#95#94#5
This fixes#116 which was caused by a refactor in commit efb6b0b9be. This bug was left undiscovered until users started upgrading from the last stable version to the current pre-release.
For an unknown reason, OBS Studio v25.x now causes a freeze to happen at this location. This should hopefully work around that issue by ensuring that we are in a location that does not cause an unusual lock order.
On highly parallel systems (> 4 Threads) audio had a chance of being played back out of order, causing it to jitter. This queue should help eliminate the issue entirely.
Fixes#111
Previously a wrong blend state caused a slight discoloration on transparent sources, which was caused by assuming them to always be fully solid without transparency. By instead relying on OBS to do the rendering we do not have to deal with blend states as much and instead can simply enjoy the result.
Fixes#104
* Remove Minimum Bitrate as it is not supported by the encoder.
* Add several default states (-1 or Default) to options.
* Possibly fix bitrate lock with CQP and CQ mode.
* Fix log output for bitrate and B-Frames.
The new logic drastically simplifies Source Mirror and reduces the attack surface for bugs introduced by humans. Additionally the new layout detection should help with improved audio mirroring which hopefully will not crash libobs as often.
Fixes#61.
This thread pool can take large or small tasks and as such alleviates the burden of having a thread per source. Particularly for large setups, this drastically reduces the number of threads running in the background waiting for work.
Fixes support for OBS_SOURCE_CUSTOM_DRAW sources and refactors the class onto better isolated and wrapped classes to deal with specific tasks. This drastically improves stability without causing code complexity to increase, and makes the code vastly easier to read too.
Related: #99
Scaling shouldn't be part of the Source and instead should be done as a filter. Not only does supporting it drastically increase code complexity, it also doesn't add anything that is really necessary as you can do everything it did better in an actual transform.
Caching wasn't actually used except for scaling and was mostly broken too, causing flickering.
Adds support for specifying Minimum Bitrate directly in the UI instead of requiring custom settings to do so. Additionally Adaptive I/B-Frames are now only shown if Look-Ahead is a value greater than 0 frames.
Quality Minimum can also now be left at a default value of -1, the Quality group is no longer toggleable and Quality Target moved into the group. Settings options on the context is now searching children too (if there are any).
Finally, some C++17 formatting was done.
Fixes#101
Scaling is now fully supported for Floats and Integers, which allows much higher precision inputs, or upscaling to a different range. Complex functions for scaling are not supported as those would be a scripting thing and should be kept as that (OBS Studio has built in Lua scripting).
Additionally, enumerations are now correctly loaded with data.
Related #5
Allow for overriding type and size of an element, opening the path for `int#[]`, `float#[]`, `int#x#`, `float#x#`, `bool#x#`, `vector<type, #>` and `matrix<type, #, #>`. Also allows for specifying the exact type of texture instead of hoping the user gets it right, as well as samplers.
Parameters are also now created if they are invisible, which means that the properties() function must not be called, but they must still be used like any other. This is due to a problem with default values not being applied all the time, and sometimes just vanishing.
The code also now throws exceptions with reasonable text, which should be caught by the gfx::shader implementation and refuse a load of the effect. No other state should be modified at that point, so care must be taken that up until the moment the complete initialization is done no other state is modified.
Thanks to the workaround in obs::tools, gfx::shader::shader now supports dynamically rebuilding the properties with new properties without crashing OBS Studio. This effectively allows you to have an up to date view of the current parameters for the shader technique.
Additionally with file watching, live development of shaders is possible at very little cost. Currently only file times and size is looked at every 333ms, but in the future it is possible to also watch for file renames and more.
The current implementation of obs_properties_remove_by_name corrupts the obs_properties_t object whenever it is called on the first or last property in the list. This leads to rapid unscheduled disassembly, and therefore must be fixed in order for this function to be used.
Fixed by upstream PR https://github.com/obsproject/obs-studio/pull/2257.
Removes the stutter when re-opening the properties dialog which was caused by recompiling the same exact shader every time this happened. Also paves the way for simple file watching.
Getting the resolution of a source is very expensive in libOBS, as libOBS does not cache it and instead always calls into the filters and sources to determine the actual source. This also leads to potential lockups due to the filter list mutex being locked for the target source.
Therefore instead of calling it multiple times, cache the result of the call, if that result is even necessary. This reduces the need to synchronize lightly parallelized work (UI and libOBS code) and helps against the potential race condition in libOBS.
Prevents the use of get() and reset() where not actually needed and forces us to actually implement all of the methods needed to interface with the object, leading to cleaner and safer code.
These two wrap the underlying gs_epass and gs_effect_technique objects, to allow direct and improved access to them without relying on the libobs API to provide this access for us. Additionally these make it safe for us to use them instead of relying on C-like code to deal with it.
This drastically improves stability and prevents all exceptions from leaking into libobs C code, which prevents crashes and unexpected freezes from exception handlers further down the stack.
Additionally minor work was done to further improve the quality and user experience for the filter.
Caching the output of a source is only necessary for really expensive to render sources, so it is disabled by default now. Thanks to that, most Source Mirrors are now "free" instead of requiring two context switches and a texture, while those really expensive can be manually set to cache.
The scaling mode is also set to disabled instead of point when rescaling is off to further improve performance. The previous method would incorrectly cause an extra texture to be used.
Additionally we now have support for debug markers for graphics debugging, allowing us to exactly tell apart improvements in rendering cost for this source.
This class and template should be used to reduce the code clutter from repeatedly doing the same thing. It requires OBS v24.0 or newer since the get_properties2 and get_defaults2 API were fully implemented with it.
This is a massive improvement to stability and safety when using the plugin, as all exceptions should now no longer be leaked into C controlled code, which can't actually handle exceptions at all.
Warnings fixed:
* Potentially throwing exception during library load.
* Possibly throwing function passed to C.
* Statement does nothing.
* Variable is initialized but not referenced.
* Variable overloads variable in parent scope.
It is extremely problematic to throw C++ exceptions into C code, especially because C code usually does not handle C++ exceptions at all. Therefore we have to prevent any exception from leaving the function and define it as noexcept.
The util::event code suffers from the problem that it could call into a class that no longer exists, corrupting memory or even crashing. By tracking lifetime using std::weak_ptr<void> this can be avoided and the dead listeners can be automatically removed.
