This summer the Synthesis team took on a couple of large challenges, with one being the transition to a virtual working environment and the other being a complete overhaul of the Synthesis backend in order to provide users and open-source contributors with a better experience. In order to tackle these effectively the interns have been working in three interconnected subgroups over the course of the past few months to bring you all of the best features of Synthesis!
In the past Synthesis has used a proprietary JSON format to store data about the shape and mechanical properties of robots when they are exported from CAD software into the Synthesis simulator. In order to move to a more flexible format the teams a variety of existing standards such as Pixar’s Universal Scene Description (USD), the XML-based COLLADA format, and a custom binary format using Google’s Protobuf library. In the end, we settled on using the GL Transmission Format (glTF) due to its simplicity, extensibility, and emerging popularity as a 3D exchange format.
We developed general-purpose add-ins for Fusion 360 and Inventor with the ability to export any assembly to a triangulated mesh in the glTF file format (*.gltf or *.glb) complete with Physically-Based Rendering (PBR) materials. Our Fusion 360 exporter add-in makes use of the Fusion component instancing system to export components that occur multiple times in an assembly just once, which significantly cuts down on export time and file size.
Since the exported files comply with the glTF spec, our open-source exporters can even be used for applications other than Synthesis. Exported glTF assemblies can be opened with any 3D application with glTF compatibility, such as Blender, Microsoft 3D Viewer, most 3D web frameworks, and more so we’re excited to see how users take advantage of these capabilities!
A large amount of this summer was spent devising a plan and executing the module system, a new feature in Synthesis allowing users to create additional functionality and extend the capabilities of Synthesis, without the need to mess around with game engines or low level code. With the advent of the module systems, users can easily add community contributions to their synthesis instance with ease. Rather than having to submit pull requests and contributions to our GitHub repository, you can simply download a module and drop it in the modules folder to start adding features.
Another key development this summer is around the idea of the virtual file system which allow users to have the power they need for proper module development, while sandboxing away elements that could be abused. We also implemented the Asset Handler, which is the one-stop-shop for accessing resources for modules as well as components within Synthesis itself.
The final thing we worked on was the Controller itself, its protocol has been implemented in Python, C#, Java and C++ so that developers have a wide array of languages to work with. With the flexibility of the control protocol, users can use the Controller for anything from implementing multiplayer to creating validation tests for their robots in the Engine.
The Engine team primarily focused on systems that we built on top of Unity to provide users with a way to visualize their robots interacting with different environments. Most of our work was done in what we call the Synthesis Core which contains all of the basic functionalities of a simulator where users can load in their robots and do simple testing.
We also developed an API that allows developers to access the functions that we use to build the simulation so that anyone can easily build their own modules to add features to Synthesis. We’ve worked to make our API as simple as possible so it’s easy to extend the simulator and add a variety of features you would like to see without waiting for our team to start a new development cycle.
This summer we also decided to integrate Unity’s new UIElements system to make it even easier to add to Synthesis. Unity’s UIElements system was brand new as of Unity 2019.1 and release of their runtime support is now labeled as TBD. Despite not being officially released, they presented the new runtime system at a Unite talk in Copenhagen. In the example they used for the talk, they included the runtime implementation of UIElements, along with a license to use it. So naturally we decided to use that for our UI system! However, because it was initially built to work in the editor exclusively, there was no current method for bringing in brand new UI from the modules at runtime so we had to spend about 2-3 weeks developing our own parser to convert XML and CSS style sheets into the UI types that unity used. During this entire process, we diddled around the line of what Unity allowed us to do and what unity didn’t allow us to do. We considered multiple times about restarting the UI system and going another route, but in the end, found that the simplicity of workflow the UIElements system worked best for us. 350 lines of reflection later, we got it working!
We hope that these changes will not only provide a smoother experience for anyone looking to use Synthesis as a simulator but also encourage developers to try their hand at extending Synthesis in cool new directions. Since we are an open-source software there’s nothing we love to see more than community contributions so we’re looking forward to seeing what you decide to do with Synthesis!