Using Popular 3D Engines and Frameworks

A trip through popular 3D frameworks for creative purposes.

If you are like me, nothing is giving you more thrills than to write or design software. The sheer amount of joy I take when coding something has led me to develop projects spanned across many software domains.

One of those domains is 3D graphics creation for Native Applications, desktop showcases, or the Web. I have never created any 3D games, but there are many times where I needed to create a 3D physics simulation or present a 3D scene in response to a stream of data received as input.

Something that no one had warned me about before I decided to develop 3D software was that it would be challenging to decide what to start with. Judging by the plethora of the frameworks intended for the Creative Scene, I had to do a lot of work before I started to do any work :).

Disclaimer: This is not another “top X” post. Some of the technologies I will talk about may surely be popular, but that’s not the point. The 3D graphics scene is vast, and this is more like a documentation of my approaches and experiences with a few solutions rather than a comparison of the top frameworks.

Game Engines

Ah, the sweet holy grail of most of the Software Engineers I have encountered.

Most of the developers I have talked to in my life have a pretty vague idea of what goes into creating a game — or even using them. 3D graphics is an entire philosophy that involves something that many software engineers dread; Mathematics. Quaternions, matrix transformations, how this applies to raycasting, how lighting works (To name a few).

After the theoretical parts, here come the practical ones: File formats (PNG, GLTF, importers), meshes, graphics specifications (OpenGL, WebGL, Vulkan, Metal). Then you have physics engines, sound engines, gameplay engines, AI engines.

Fortunately, we have Game Engines to help us abstract away from all those annoying things, lifting the developer’s burden and allowing us to concentrate on what matters: our game. But what if we don’t want to build a game? What if we wish to run a 3D simulation, visualize some data, view some models, or work in a simple AR application?

AAA Game engines

In the game development scene, you have those big game engines capable of going everything out of the box, like Unreal Engine, Unity, CryEngine. These engines can do absolutely anything. They were initially designed with building games in mind — and it shows. Some of the best AAA games are built with those. There is also the Godot engine.

I began my trip with game engines a while ago using my Mac. I know that gaming on a Mac has been an afterthought for years (both for creation and play), but not developing and playing on a Mac is a deal-breaker for various reasons. First of all, it shows how well-thought is a game/game engine. Secondly, my main development machine is a Mac, and I don’t intend to abandon it any time soon. Lastly, AR shines on iOS, and with Metal, macOS has been turned into a processing beast. In any case, consider my experiences here to have been acquired using a Mac (though the nature of the incidents should make no difference to the reader).

AAA engines come at a cost, either in Royalties or in a per-month fee. However, the ones mentioned offer some very permissive free plans that allow experimentation and usage in production.

From my limited experience with Unity and Unreal, I would say that Unreal wins in the “best looking game engine” category while still giving the ability to develop on a Mac (CryEngine cannot do this at the moment). Unity, however, wins definitely in the “best ecosystem” approach. It was relatively easy for my simple 3D game to find anything I needed by performing a simple Google search. I believe that the programming model of Unity has something to do with this. You see, in Unreal, you will either go with Blueprints (Unreal’s visual scripting system) or C++. Unity also offers a Visual scripting system (I don’t believe it’s as full-featured as Unreal’s) and C#. C# is a far easier language to learn and use than C++, making development much more attractive to less experienced programmers.

It’s no wonder that most job applications for game development mention Unity as a prerequisite.

For the enthusiast: Godot
Lately, however, there is one more game development engine that drew my attention, named Godot. Godot touts itself as the “game engine we had been waiting for” and introduces a “different way to make games.”

I was intrigued, I decided to give it a go, and I was impressed.

Godot takes approximately 75Mbytes in my Mac, including the entire application, with the game engine included. And it’s completely free and Open Source. No royalties, no payments, not even a simple registration.

Godot’s development model revolves around putting behaviors and appearances into objects like animations and scripts and reusing them wherever you like. This was pretty new to me- and it just “clicked” into my mind. This was the way I wanted to develop my scenes. Not to mention the excellent-looking user interface that makes you want to use the engine. Godot’s scripting language is mainly GDScript, a fully capable python-like language, and support for C++ (through bindings) and C# (added in a later version) is also present and fully supported.

Godot is rumored to excel in 2D game development, but not in 3D. For the scenes I created and the demos included with the engine, Godot is more than enough, although it cannot perform in complex 3D scenes and the AAA engines. That said, the Godot rendering engine is being rewritten to use Vulkan (and on OS X, it will use MoltenVK as a translation layer from Vulkan to Metal), so Godot 4.0 will shorten the distance in performance and quality from the AAA engines.

All in all, Godot is the game engine I enjoyed the most while developing my hobby projects. GDScript is fantastic for game development. Godot is so developer-friendly that someone has even created a sprite editor in Godot using nothing but the engine itself — Pixelorama.

Overall, if I were going to invest time into game development to be hired by a big company, I would choose Unity and still learn Godot. For my current projects, which are nothing more than simple proof of concepts or simple simulations, Godot is more than enough and will become better.

Honorable mention: Slightly off-topic, but if you are looking for an excellent 2D game engine, have a look at Cocos.

Graphics Frameworks / Middlewares

Having game engines such as Unreal, Unity, or Godot may seem enough for any project involving 3D graphics.

This is not true. Although those engines are more than capable of producing the kind of graphics fidelity I was interested in, I have decided not to use them for my projects requiring 3D graphics. For many use cases, using a full-featured game engine may seem overkill.

I approached those engines with the following mindset:

• I am a hobbyist 3D developer (note that I am not a game developer)
• I wanted to create 3D graphics, and integrate them well with sound and make them interactive.
• I wanted to be able to perform simple simulations using the default physics engine.
• I wanted to incorporate whatever I built into a native application, whether on Windows, iOS, Android, or Desktop.

When it comes to researching solutions for Creative Coding, the first choice I needed to make was whether I would develop something for the browser (and I could embed it anywhere) or for the Desktop. I have researched both ways of doing what I needed.

Pros of Web 3D:

• Much easier to set up
• It does not require you to mess with native compilation — which is a pain because of differences between OSes and their drivers and OpenGL implementations.
• You don’t have to worry about OpenGL being deprecated (especially if you want to address macOS users).
• There are much more developer-friendly frameworks than Native
• Javascript is much more approachable than C++ (which is the language of choice for most native 3D libraries and frameworks).
• It’s cross-platform by nature. It can also be embedded into any web application effortlessly.

Pros of Native 3D:

• Extreme performance. There is no limitation to what you can do. And if OpenGL is not enough, you have Vulkan or Metal.
• You have much better access to the computer’s peripherals. This is important for creative coders doing work on external inputs.
• It has multithreading. This is not supported in Javascript.
• You have many more choices (although the majority of those choices require you to go much deeper than if you were developing for the browser)
• C++ is handy. No, really. Like when you want an expressive API to perform mathematical calculations. Operator overloading really shines in C++ and GLM.
• I know we are talking about 3D — but if you want to associate your visuals with sound, there is simply no comparison between the Native and Web libraries, so depending on the amount of sound processing you will need, you will have to go Native.

Ultimately, the needs of my projects dictated the path I should take. The gist of all this is that the Web comes with its own set of limitations and makes it up for you by providing development ease and simplicity. I would say to begin with the Web approach and then go deeper in Native if you want to take advantage of peripherals, sound/video processing, etc.

Native Creative Coding Frameworks

OpenFrameworks

OpenFrameworks is a perfect solution for the creative coder. It’s an open-source project based on C++, and its documentation is excellent. OpenFrameworks has been around the longest compared to other creative coding frameworks, and its community is much larger as a result. Much of OpenFrameworks’ shining capabilities can be attributed to the vast add-on ecosystem it has. Some are more well-maintained than others, but it sure shows that OpenFrameworks is a well-supported framework for building the foundation of your solution. When it comes to supporting external peripherals like Kinect, there is a high chance that someone has released a library for it.

Consider OpenFrameworks if:

• You want a performant creative-coding framework using C++
• You need to take advantage of the huge community and need any of the great extensions the community can provide.

Cinder

Cinder is an alternative to OpenFrameworks. It has a much smaller user base, but it is targeted towards developers who want a robust and Object-Oriented approach. OpenFrameworks has the approach of wrapping C-style code and libraries with the C++ language. Cinder, on the other hand, is built around the concept of being used with C++.

That makes Cinder a very well-thought framework, with a great programming paradigm that aims to be taken advantage of by the seasoned developer. It has very good documentation and had built-in support for Audio, Video, and Image Processing. It also provides a rich mathematics and utility library as part of its core.

Out of the things that I liked very much when working with Cinder is how easy it was to create something from scratch while at the same level being able to get as close to the metal with it. Cinder does not make too many assumptions about what you want to do.

There is a showcase page if you want to find out more about some projects developed by Cinder.

Who should use Cinder?

• Creative coders looking for an alternative to OpenFrameworks
• Developers looking for a more C++-centric approach to creative coding.
• People who are not afraid to try out new things and live with a framework with a smaller community

Honorable mention 1: Magnum Engine

Magnum Engine is a framework that drew my attention as one of the low-level choices for cross-platform data visualization.

I didn’t end up using it as my projects always required something at a more high level. I couldn’t help but be impressed with the work being done to provide a platform for graphics, not just ready-to-use libraries.

Magnum is one of the frameworks that one would use to develop their own solution or library on top. OpenGL is supported, and Vulkan is currently being added. If you like investing time into lower-level graphics mechanics, this is the framework for you.

Honorable mention 2: bgfx

bgfx is a cross-platform rendering library. It is low-level and C++-oriented. bgfx supports a wide array of rendering backends and platforms: Direct3D, Metal, OpenGL, Vulkan — all those for iOS, Android, Linux, Windows, etc.

The work done is amazing, and it is one of the very battle-tested low-level graphics libraries out there. Many games use this as their rendering backends (like ioquake3, mame), and its capabilities are limitless. Fun fact: it’s also used by Babylon Native (read below)

bgfx is very low-level, it provides a nice abstraction layer on top of rendering frameworks and platforms so that you don’t have to think about supporting different vendors and codebases.

As such, it is not much intended for creative coding, but that doesn’t mean that you shouldn’t use it. You should be aware that if you intend to use it, you will want to go deeper than what you would normally do with solutions such as OpenFrameworks.

Web Creative Coding Frameworks

If you are searching for a way to do 3D on the Web, things are simpler there. I have only used two of the available engines on the Web, but with those engines, I really didn’t need anything more than what they had to offer.

ThreeJS

ThreeJS is the most popular 3D graphics framework on the web. It has a website full of 3D examples. Its community is vast, and there are game engines built on it, too. That’s because ThreeJS is not a game engine per se. In fact, it does nothing more than rendering graphics.

It does so more than adequately, however. People have been building beautiful things for years with it. I have used it for simple projects, where I tried to make simple GLTF viewers for a specific use case I was having. I found that I could easily create 3D scenes, put some stuff in them, put some lights, and a camera. The graphics quality was also outstanding.

ThreeJS comes with an editor to build a basic scene or test an existing one; there are many exporting options. You can create a scene end export to ThreeJS native format, COLLADA, GLTF, wavefront obj, and many more. Overall it was great working with it until it came to a point where I needed something more.

Babylon.js

I must admit that I wasn’t sure if Babylon.js should be placed into this category, as it can be used both as a game engine and a creative coding framework.

Babylon.js has been released in 2013 and was developed by two Microsoft employees, and it still is backed by Microsoft. Babylon is a full-fledged game engine, which goes beyond just having a creative coding framework or a rendering engine. It has also been expanded to cover native platforms as well — although it’s too early to tell how that will pan out. Babylon Native uses bgfx as its rendering backend.

Babylon.js comes with a great playground tool to allow some real-time coding and scene construction. It also includes a sprite editor, a material editor, end much more. There is also an excellent editor, which is maintained by the community and is open source. It supports drag-and-drop materials and objects, setting up scenes with advanced lighting effects, putting sound effects, building game logic, and much more.

In my experience, Babylon has rendered the scenes that I had put to it using more natural lighting, and it was also able to handle larger scenes with hundreds of thousands of objects without stressing too much.

Although you can use the Editor to create Babylon scenes, you can also use the plain javascript libraries to do this by hand. This is very important because I wanted to load some GLTF models dynamically and put some lighting and effects based on the imported models. I wouldn’t be able to do this with prebuilt models. Setting up a camera is also very straightforward.

What makes me prefer Babylon from ThreeJS is the ease of use, the concrete API it has, first-class Typescript support, and the excellent toolchain.

Conclusion

I wanted to document my experience using some 3D engines/frameworks over the years through this post. Creating 3D graphics is fun and challenging, no matter your experience. It’s also the most rewarding for me, although I don’t currently have a day job supporting those kinds of projects.

Overall, it seems as if the 3D graphics software scene is full of solutions, and there is no one-size-fits-all approach. What helps most is to understand the underlying theory and technology of 3D graphics, and then approach each framework/engine with the mindset of solving a well-defined problem each time. Therefore, you should expect to do a lot of work to identify the 3D software tools for each occasion.

Interesting links:

• Awsome Creative Coding libraries and frameworks collection
• Good comparison (a little outdated) of more graphics frameworks. https://dev.to/funatsufumiya/comparison-of-c-low-level-graphics-cross-platform-frameworks-and-libraries-58e5
• Ogre3D — Open Source 3D Graphics Engine
• Diligent Engine — another interesting 3D engine