.png)
Have you ever played a video game or watched a movie and been so blown away that you heard yourself saying, “WOW! I wonder how they did this?”
Most of the time, these rich visuals are created using a combination of Flutter shaders. Take a look at the visuals in this article, where shaders play a critical role. Today, you’ll learn about a specific type of shader called a Fragment Shader and how to use it in your Flutter application. Outside of specifically discussing Flutter Fragment Shaders, I’ll touch on many other things including giving you some basic background on shaders for those who may not be as familiar.
With that, let’s get started!
What are Shaders?
Shaders are a set of instructions that a system executes for every single pixel on the screen. They are responsible for manipulating the color value including light and darkness for that pixel during rendering. Instead of painting layer by layer, you paint every single pixel on the screen, giving you more control over your painting.
These instructions run for every pixel in parallel; thus, shaders in Flutter require quite a lot of computational resources to run smoothly. It is also important to note that shaders run on your GPU (Graphical Processing Unit), which is specially made for handling tasks like manipulating pixels and images. Because of this, shaders are extremely fast.
How to Write a GLSL Shader
There are multiple languages you can use to write shaders, including GLSL (OpenGL Shading Language), SKSL (Skia’s Shading Language) and MetalSL (used by Apple devices), among others. If you’re familiar with languages like Dart or C, you should be able to pick up these languages fairly quickly. For the sake of this tutorial, I’ll stick to one language, GLSL.
For a gentle introduction to writing a GLSL shader, we will be coding a simple example of a shader that outputs the beautiful gradient displayed above.
Let’s go over the code step-by-step :
- Like Dart, GLSL defines a single entry point through the main() function. At the end of this function, you set the color for the pixel.
- GLSL has built-in variables like gl_FragCoord and gl_FragColor. gl_FragCoord gives you the pixel vector position, and you set the pixel color through the gl_FragColor. When you specify a vec4, you’re actually setting the rgba values of the color for that pixel.
- GLSL allows us to create conditional compilation blocks and defining values in the pre-compilation stage. These blocks or values start with #. #ifdef(if defined)#endif is a conditional operation block that is checked before compilation. The 2nd line executes if the GL_ES is defined, which is a version of GLSL for embedded systems like mobile devices and video game consoles.
- Level of precision is everything when you’re dealing with colors. The more precise you are, the better quality you get. The 2nd line sets the float precision to mediump. You can also change it to lowp or highp if you want.
- Inputs for GLSL shaders are defined by adding a uniform in front of them at the top of your code. One of the most important inputs is the u_resolution (also called the iResolution) of type vec2 which gives you the resolution of the screen.
If you want to learn more about Flutter shaders and how to write them in detail, check out The Book of Shaders.
Implementing Shaders in Flutter
With the release of Flutter 3.0, initial support for creating custom shaders was moved to stable. Therefore, it’s important to make sure you’re running the latest version.
You can upgrade to the latest version by running the following:
If you’ve been around Flutter for a while, you may already know that Skia is the Flutter rendering engine that is used for rendering UI and SKSL is its shading language.
Support for Flutter custom shaders is through the SPIR-V Dart API, which was moved to stable in the release of Flutter 3.0. SPIR-V is an intermediary language designed to act as a bridge between different shading languages in order to make it easier to port shaders from one language to another.
To use custom shaders in Flutter, you convert your GLSL code to SPIR-V binaries. (Learn more about this.) Once you have the binaries, you need to use the FragmentProgram from the SPIR-V library, which helps to compile the SPIR-V binary to SKSL and to create a shader from it. However, this manual process is quite time-consuming and not that intuitive if you’d like to get started quickly with Flutter shaders.
To make things easier, we’ll be taking a look at a tool that will automate this process and generate the SPIR-V binaries from our GLSL code, as well as provide an intuitive API to learn GLSL shaders in Flutter.
Support for writing custom Flutter shaders is a work in progress and not yet marked for production.
Compiling Shaders Using Umbra
To make your life a little easier, there’s an amazing tool from the community called Umbra that handles shader compilation and creation for Flutter.
Umbra is a CLI and visual editor for writing Flutter shaders. It comes with an abstraction on top of the Flutter API to make the lives of developers that want to write shaders easier. The main goal, therefore, is to make writing shaders in Flutter easier, faster, and overall more enjoyable. — Umbra Docs
The Umbra CLI takes care of the compilation of GLSL to SPIR-V and generates the Dart API. The API handles the compilation of SPIR-V to SKSL and provides intuitive, strongly typed language support for your shader. It also adds some abstraction on top of your GLSL code which helps you focus on writing your Flutter shader code. It also adds the necessary raw GLSL code before it compiles the shader. You’ll learn about the abstraction Umbra adds in the next section.
1. Activate the Umbra CLI.
2. Once activated, download the Umbra dependencies.
This might take a while, so sit back and relax a bit!
Creating Your First Flutter Shader
You’ll learn how to add this cool shader, which kind of resembles the waves in water, but a bit fancier. To create it, follow these steps:
1. Add the umbra_flutter package in your pubspec.yaml
Or add it with the CLI by running:
2. Create a new shader project with the Umbra CLI by running:
This will generate a GLSL file at the root of your project where you can add your GLSL code.
3. Add the following GLSL code to the hello_world.glsl file
As you can see, some of the configurations we saw in the earlier example of the GLSL code are abstracted by Umbra.
The main() function is replaced by a fragment function which gives us the uv (normalized coordinates of the pixel) and fragCoord (coordinates of the pixel without normalization-gl_FragCoord), and returns the pixel color instead of assigning it to gl_FragColor.
We also added another input called u_time, which represents the time since we started. It’s crucial when you want to animate your Flutter shaders with respect to time.
4. Create a folder named "shaders" in your lib folder. Now, run the following command, which takes the path of the GLSL shader file and the output location:
This generates the Dart files and the SPIR-V binaries at the given output location for your shader.
5. Create a my_shader.dart file somewhere in your lib folder. Add the following code within that file and import your HelloWorld shader in this file:
What you’re doing here, in short, is compiling your shader. As the process is async, you use FutureBuilder to load it. Once it’s compiled, you then use the .shader() method on your UmbraShader object and pass it the required inputs.
The resolution is the size you want for your shader, and the uTime is the delta (time since we started). delta is calculated with the help of Ticker. Ticker triggers a callback for every frame that increments the delta by 1/60 ms.
Finally, you pass the created Flutter shader to the CustomPainter to paint.
If you’re going to create shaders often, I would recommend saving the above code snippet in Pieces and later updating the shader's type and inputs.
Now that you’ve built everything, run it! Your results will end up looking like what is displayed below.
Awesome! You just created your first shader in Flutter! Isn’t that exciting?
The source code for the project can be found here.
Limitations of Flutter Shaders
There are a few limitations and constraints when writing your GLSL shader code for Flutter. Some of the limitations are from Skia, and some are related to the current integration of shaders into Flutter itself.
The following are some of the specific limitations of the SPIR-V library:
- for loops must initialize a float variable to a constant value, compare it against a constant value, and increment/modify it by a constant value
- It does not support while and switch statements
- The types that are supported sampler2D, bool, float, float-vector types, and square float-matrix types
- It only uses the built-in functions present in GLSL ES 100
These are some of the limitations/rules mentioned in the SPIR-V library that you have to adhere to while writing your GLSL code. Some of these rules are handled by Umbra while generating the raw GLSL code, but for some of them, you should be on the lookout.
In addition, note that there isn’t Flutter web support for shaders at this point. These limitations will likely go away in future versions of the library.
Next Steps
The initial release for supporting Flutter shaders paves the way for some creative coding and unlocks new and never-before-seen experiences.
While I researched shaders in Flutter, I found some interesting features on the Flutter GitHub that may land in the near future. Some of them add support for compiling your GLSL shaders to SPIR-V through the Flutter CLI. There are even proposals that would make it possible to create custom shaders and ColorFilters with SKSL, which could be extremely useful for image/video editing apps and games in general.
If you are interested in staying up-to-date on new developments in shaders or reading more technical articles like this, be sure to subscribe to the Pieces blog to get updates on weekly articles and check out Pieces, the fastest, smartest AI Assistant for code snippets and screenshots.
An Easier Way to Save Your Flutter Fragments
When developing Flutter applications, you may have tons of widgets you save that you want to reuse later, but you just do not have them in a safe place where you can access them. There also may be the scenario where you are combing through Flutter and Dart documentation, and you want to save examples that come in handy when implementing a new feature or figuring out which widget to use for different circumstances.
Pieces helps you save all of your useful code snippets efficiently through a desktop application and integrations. Using Pieces, you can save any code snippets from StackOverflow with the click of a button using the Chrome extension, have your code autosaved from locally-hosted ML algorithms that recognize your code patterns, auto-classify snippets by language, share code with others using generated links, and more! The Pieces for Developers Suite is continuously being developed, and there’s some groundbreaking stuff that is being put together to share, reuse, and save code snippets.
