The high priority CUDA stream causes libOBS to be at a lower priority than the tracking, which is not what we want. Instead we want tracking to be incomplete in those cases, rather than slowing down encoding and other things.
Geometry updates are also now done once per frame instead of one per tracking update, which should improve the smoothness without affecting performance too much. Additionally all tracking info is now in the 0..1 range, which drastically simplifies some math - especially with texture coordinates.
To deal with tracking and updates being asynchronous, a very simple approximation of movement velocity has been added. This is mostly wrong, but it can bridge the gap where tracking updates are slower, as the values are all filtered anyway.
The new libOBS API allows us to directly access the underlying API instead of having to mess around in memory. By using it we can avoid crashing in case the compiler for it is different, or in case the actual back end structure changes.
Additionally the mostly unimplemented and unused options have also been removed, which streamlines the use of this class even further and reduces both shader and code complexity.
Finally by optimizing the use of the internal render target we can achieve a speed up of up to 3000% over the old way, allowing for many more mipmapped filters.
* Fixed 'Pixelator's color transition point being off-center and uncontrollable.
* Fixed 'Drunk' filter not working at all.
* Added an inverted mode to 'Luma Burn'.
* Added exponential Luma to 'Luma Burn'.
* Fixed odd color behavior in the 'Color Shift' transition by switching out HSL with HSV.
* Added a new 'Sliding Bars' transition shader, for an example of it see this clip: https://clips.twitch.tv/RacyEndearingHorseradishAMPTropPunch .
As OBS Studio locks some mutexes in a different order depending on what actions are being done, using modified_properties for GPU work causes things to freeze in place. Instead have users manually click the refresh button when they changed files in order to prevent this freeze from happening.
Fixes: #118
Ever wished you had a professional camera operator to highlight and follow the action, ensuring the audience never misses a beat? Thanks to NVIDIA, you can now do this at home for free! The new NVIDIA AR SDK unlocks augmented reality features, including motion tracking for faces.
This allows me to provide you with an automated zoom and cropping solution for your video camera to transform your streams into a slick, polished broadcast, where you’ll always be the star of the show. Don’t forget - everything is customizable so the possibilities are endless. You can even recreate that Futurama squinting meme if you wanted to (with some scripting)!
The filter requires compatible Nvidia RTX hardware and the Nvidia AR SDK Runtime to be installed ahead of time. This filter is considered "stable" and shouldn't change much from version to version.
CRT Curvature: Emulate an old CRT TVs curvature ...
CRT Scanlines: ... and emulate an old CRT TVs scanlines, rollbar and bleeding!
Hexagonize: Turn things into hexagons. You know, like the thing you see in my streams.
Color Shift: Uses HSL and some math to smoothly shift Hue Saturation and Lightness between A and B.
Luma Burn: "Burns" away the Luminosity of A to reveal the B side.
Pixelator: Classic/Retro pixelation effect often seen in older games to transition between settings.
Adds support for specifying Minimum Bitrate directly in the UI instead of requiring custom settings to do so. Additionally Adaptive I/B-Frames are now only shown if Look-Ahead is a value greater than 0 frames.
Quality Minimum can also now be left at a default value of -1, the Quality group is no longer toggleable and Quality Target moved into the group. Settings options on the context is now searching children too (if there are any).
Finally, some C++17 formatting was done.
Fixes#101
This drastically improves stability and prevents all exceptions from leaking into libobs C code, which prevents crashes and unexpected freezes from exception handlers further down the stack.
Additionally minor work was done to further improve the quality and user experience for the filter.
Caching the output of a source is only necessary for really expensive to render sources, so it is disabled by default now. Thanks to that, most Source Mirrors are now "free" instead of requiring two context switches and a texture, while those really expensive can be manually set to cache.
The scaling mode is also set to disabled instead of point when rescaling is off to further improve performance. The previous method would incorrectly cause an extra texture to be used.
Additionally we now have support for debug markers for graphics debugging, allowing us to exactly tell apart improvements in rendering cost for this source.
Adds a new property to control the alignment of the source within the calculated boundary when rescaling the source. Also fixes the permanently left aligned mirror at the same time.
Fixes the bug that rendering an outline would remove all other effects not directly under the outline, while also improving the rendering speed drastically by only initializing and clearing the rendertarget once instead of for each effect.
This filter allows the use of another source as a mask, allowing complex filter graphs and trippy effects, such as creating a text source with three animated videos, each using a different color channel as the mask.
Fixes#18.
Automatically updates the long description (hover text) if the Type or Subtype field is changed, allowing for more contextual information about what the selected information does.
This refactors the SDF Effects to use a normal blend function instead of doing the blend in the effect itself, improving quality and reducing problematic sampling issues. In addition to this, the effect files have been cleaned up slightly and renamed to their proper names. Glow and Stroke are now supported, which solves both #2 and #4 in one go.
The caching optimization has also now been implemented, reducing the number of renders for this filter to 1 for each tick.
Dual Filtering (or Dual Kawase) is an approximation of Gaussian Blur that can reach much higher Blur sizes at a much lower cost. However it is locked to a 2^n size, which means that currently it isn't possible to use it for blur sizes like 19, 24 and 31.
The Blur works by using the linear sampling of a GPU, combined with down- and upsampling and carefully placed sampling points. This means that there is no need for a linear optimized version of this Blur.
Related: #45, #6
While Gaussian Blur is not a Blur type that really benefits much from linear sampling, it can be used for a slight quality and gpu usage reduction. However for Area and Directional Blur there is a better alternative: Dual Filtering Blur. And as with all other currently implement Linear versions of Blur, only Area and Directional Blur are supported.
This type of Gaussian Blur also has the loading hitch that exists in normal Gaussian Blur.
Related: #45, #6
Gaussian Blur is another Blur that now supports the new system, increasing the maxium Blur size to 128 and adding support for Rotational and Zoom blur. Various optimizations were done to the actual shader code which further reduced the GPU usage.
Currently the Gaussian curve is recalculated when the blur is first created, which can lead to a short hitch due to it having to search for the correct kernels. This is currently unavoidable and expected behavior until a better solution is found.
Related: #45, #6
Box Blur is a prime candidate for Linear optimizations, and as such reducing the total necessary samples by about half. However due to the reduction in samples, only Area and Directional Blur are supported.
Related: #45, #6
Box Blur is the first to be on the new system and has received all the new features and optimizations available. The maximum Blur size has been increased to 128, Rotational and Zoom Blur are supported and a small optimization has been done to the shader.
Related: #45, #6
Earlier versions tried using Trilinear as the Filter, but the correct name is Linear. With this the sampling is now set to be Linear instead of undetermined.
Additionally the logic for the displacement was adjusted and now no longer renders incorrectly when going from texel to texel. This was due to the unstable math being performed to retrieve the sign of the number, but the sign() command can do it without this math being required.
This works around the incorrect shadow on the edge of the Signed Distance Field, but it does not get rid of it completely. Due to linear sampling it will still show at some point so a different solution has to be found in the future.
Colored shadows previously had an effect on the image color, but this is incorrect if the image is fully opaque. This fixes it by using premultiplied alpha mixing instead.
This allows increasing or decreasing the accuracy of the Signed Distance Field at runtime without requiring the initial source to be resized. Use case for this would be small text where the higher quality would only be noticable on the shadow and not the text itself.
Sources were rendering with a black border around them if they had a soft fade, which is due to how the shadow calculated the image sample. With the new shader code this is now fixed and the source looks like it should be. Additionally this removes the bug where enabling any shadow would cause only the texels to draw that were above the threshold.
Additionally this adds support for inverse gradients (min > max) and negative gradients for outer shadows instead of only positive gradients. This technically allows for cleaner shadows.
The filters are always the same for every language and should not be translated, instead only the file type should be translated. This way bad translations will not affect the options that the user can select.
Rendering will now use a cached version of the mirrored source in order to reduce rendering cost drastically for heavy sources or scenes. Additionally rescaling now uses the internal scene instead of being a custom implementation, allowing for some new things.
Rescaling now has a new option: Bounds Type! This option allows control over just how a source is scaled, by default it is set to Stretch, but there are other options that can keep the aspect ratio of the mirrored source in tact.
Additionally the ScalingMethod struct has been replaced with obs_scale_type, which means that Bilinear Low Resolution is now no longer an option.
Implements #35
Directional Blur (also known as Motion Blur in some art software) enables full control over the direction the passes are applied in, which can further be combined with the scaling option. With this very simple blur effects can have a massively bigger influence.
Step Scaling allows for improved control over individual Blur passes and can be combined with Directional Blur to imitate certain effects. It also allows smaller Blur sizes to cover a larger area, although with a drastic loss in final quality. For example, if someone is streaming at 720p and wants to have their dynamic 2160p background blurry to a very high degree, this option would allow that at next to no quality loss in the final stream. The actual values that need to be used vary for each source and encoder however, so they will have to be chosen carefully.
This speeds up Gaussian Blur and Linear Gaussian Blur drastically reduces time spent reading textures and instead uses existing registers - maximizing time spent reading the actual image texture.
See Also: #21 Blur Quality
Similar to Linear Box Blur, this version of Gaussian Blur reduces the total number of sample by up to n. This results in a total sample count (per pass) of O(n+1) for even radii and O(n+2) for odd radii. The quality sacrificed to do this is higher this time, though careful adjustment of the halfTexelDelta value can bring it much closer to normal Gaussian Blur. The current offset however had no noticable effects on visual quality.
See Also: #21 Blur Quality
Linear Box Blur abuses the fact that with Linear Sampling we can sample up to four adjacent texels at the same time and get a correct result for Box Blur back. Using this the total number of sample for Box Blur is reduced by n, making the total either n+1 (Even Radius) or n (Odd Radius).
Additionally all blur effect files have been merged into a single blur.effect file to reduce the time required to change a single parameter name. New blur effects should be added as a new technique instead of as a new effect file.
See Also: #21 Blur Quality
Adds Inner/Outer Shadows for dynamic sources based on signed distance field generation. This is fast, but does add a bit of latency when it comes to updates - which means that moving objects will leave a trail before the generator has a chance to update.
Fixes#3
* Removes the old 'Region' fields and places them under a 'Mask' option that can do much more.
* Supported Mask types: Region, Image, Source.
* Image and Source mask types allow for a color filter and multiplier.
* Supports region, feathered region, inverted region, inverted feathered region and image input.
* Interpolates between image_blur and image_orig depending on the result of the mask function.
This enables full mipmapping support for textures with a shader view that allows accessing different mip levels. In order to access mip levels, you have to specify gs::texture:🎏:BuildMipMaps when creating the texture, as OBS currently forces the maximum mip level to 1 even if you actually have mip data available.
The Custom Shader Filter is now capable of rendering a custom shader using the gfx::effect_source class as a backend. An example shader is provided for starting off, more advanced examples may come later.
Related: #5
Rendering is now possible, although some parameter types are not yet supported. So far, booleans and floats will work fine, integers will cause an error in OBS Studios rendering code for an unknown reason.
The Blur Filter can now be applied to a region inside the source itself, the inverse of that region, and/or a feathered version of that region. This allows for easier scene setups where only some parts need to be blurred, but the rest can be left as is.
Fixes#12
From this point on, Source Mirror is now capable of real-time mirroring of Video and Audio. This can help if you need different filters per scene for your microphone or voice chat, depending on the scene (audio ducking for pause scene, no audio ducking for live gaming).
Custom Shader allows you to write your own effect files and just have them applied to your source(s). It will dynamically update the properties to match the parameters in the source as well as offer some special parameters to the shader.
# Conflicts:
# CMakeLists.txt
# data/locale/en-US.ini
# Conflicts:
# data/locale/en-US.ini
Scaling was previously incorrectly rendering the source with another effect forced onto it, resulting in slower rendering and some sources that would no longer render properly.
Additionally the new option allows the user to have the source render at the original resolution in order to allow previously applied transform to stay identical. The rescaling however will no longer apply to filters after this source then, thus the speed bonus is lost.
Also categorized the localization file and adds descriptions for existing and new properties for Source Mirror.
The source allows you to apply effects to the same source without requiring a new instance of the source. Any changes done to the original source also apply to the mirror, so there is less total work that needs to be done.
It can also rescale the source, allowing you to use the same source as a cheap instant backdrop with Blur for example. What you do in the end is completely unwritten and up to you to decide.
End goal of this is to be an anisotropic filtering purely on the GPU without any actual mipmaps in the graphics subsystem. The basic idea behind it is that we can create all possible mipmaps in a twice as big texture as the original and then can efficiently render it no matter the GPU features as it will only consume one sampler.
The texture that it outputs will have both horizontal, vertical and full mipmaps in it. Horizontal mipmaps reduce Width, Vertical mipmaps reduce Height and full mipmaps reduce both. This means that LoD levels are both possible in the X and Y direction, allowing for much greater precision mipmapping on all GPUs.
This is a temporary solution until we can directly copy updated textures to a mipmap level.
This option allows applying the blur to other color formats such as YUV. Bilateral Blur benefits the most from this, resulting in smoother images at lower kernel sizes.
Michael Fabian 'Xaymar' Dirks
