Web Tools for Application UI

Yesterday at GDC 17, Andreas Fredriksson from Insomniac gave a fascinating talk about their experience using web tools (HTML, JS) to create AAA game editing tools. You can see the slides here: https://deplinenoise.wordpress.com/2017/03/03/slides-insomniacs-web-tools-postmortem/.

I believe that using web tools for application UI development is a good solution. At Coherent Labs, along with our main middlewares, we develop the Coherent Editor, which is a game UI development tool written with web tech, like Insomniac’s.

I want to give a sum-up of our experience developing the tool and how it compares to Insomniacs’. The Editor is still in heavy development and we are happy how it’s shaping up and the development process we built around it.

The Coherent Editor in version 1.5.3.4 looks like this:

coherent_editor

The final output is a game UI in standard HTML/CSS that can be consumed in Coherent GT or Hummingbird. This particular sample will look like this in the game:

data_binding

The Coherent Editor is currently ~100K lines of JavaScript, it’s smaller than Insomniacs’ tools, but still a very large application by standard-JS-app sizes.

JavaScript gives us the same benefits that Andreas has highlighted + some more:

  • Super-quick iteration
  • We can leverage JS libraries
  • Exercises our own runtime – Coherent GT
  • Can output optimal HTML/CSS for our runtime
  • Directly interfaces with our products
  • Can have game-specific features

Finding great JavaScript developers seems easier than C++ devs. We leverage all C++ devs only for the development our runtimes.

Andreas highlights many issues they encountered while using the web stack in their development process, I’ll sum-up how we solve them in our team:

  • Don’t use Chrome/CEF/Firefox/Edge etc.

The Coherent Editor runs on-top of Coherent GT. Here we have the advantage that we create our own technology. 5 years ago when we started Coherent Labs, we based our first generation product on chromium (the project Chrome builds upon). We quickly realised this is not so good. Chrome is an amazing beast but does too much things, changes too often and breaks things all the time.

With our own technology we keep features stable and concentrate on what makes sense for games. Generally GT is an order of magnitude faster rendering-wise compared to Chrome. Chrome has a super complicated out-of-process architecture that is useless for games and hogs memory like no other.

Andreas notes that in the end they had to freeze the Chrome version they used and that a stable custom runtime would have been better. When we started the Editor there were debates if it should be based on Chrome/Firefox, but fortunately we didn’t go that road.

  • Extend communication JS <-> C++

We have a very easy layer of data binding in GT that allows communication and data-exchange between JavaScript and C++. All the OS-specific things is directly implemented in the Editor. An API is exported from C++ to JS and it can do everything it needs – loading/saving files, launching external tools, importing from Photoshop/Illustrator and so on. This removes a huge amount of complexity, when you need something written in native, you simply go, write it and use it from JS.

Performance-heavy tasks can also go to other threads in C++, we are not limited to the threads JS allows.

  • Test all the things

We also quickly realized that a lot of JS code becomes quickly very brittle. Fortunately JS is also easy to test. We added Selenium support to Coherent GT and now the Editor team writes test for all their features, which helped a lot for the stability and confidence in the tool.

  • Use declarative data binding and components

The best code is the one that does not exist. GT has a declarative data-binding layer that allows UI developers to NOT write any JS code and attach fields, properties etc. directly to the C++ data model. It can also instantiate UI elements (components) in a “for”-cycle way.

Think an “Open file” dialog, where the “model” is the list of files read from C++ and the dialog is directly populated with the “File” components in the UI. The “File” component is a small piece of HTML with and icon, text, properties etc. This approach saves a ton of time and reduces the risk of errors.

  • Typescript

The Coherent Editor is written in Typescript, which is also what Insomniac ended up doing. Specialized scripts run autonomous and recompile and redeploy the changed JS. It happens pretty quickly at least in our application and doesn’t seem to impact iteration time significantly. I guess it depends on the overall project structure though.

  • GC configuration

We have configs on the JS GC that make it run less often during heavy tasks and allows it to do it’s work when the Editor is relatively idle. It increases the peak memory usage, but eliminates hiccups.

Conclusions

Using web-only tech with a standard browser is very, very tough to get right for a large Application UI shell. What saved us is that we have our own technology that solves the downsides of a “pure” browser-based solution.

  • Performance was not a problem – GT is a game UI runtime designed for consoles so on development machines, the Coherent Editor UI flies.
  • Data-binding really makes a difference. We designed it to make it easy for games to communicate with the UI and in essence NOT write any JavaScript. Separating the UI development from the “backend” speeds up development x10.
  • If something doesn’t work as we like – we can change it. Although Chrome is OSS, making changes there is incredibly time-consuming and risky (they break it the day after). Believe me.. I’ve gone that path.

We still have a lot of work to do on the Coherent Editor. There are operations that can put it to it’s knees (try loading 4000 images and go grab a coffee) but they are mostly related to handling specific cases than to the overall architecture. We haven’t yet achieved the complexity of Insomniacs’ tools and I hope it’ll hold when we do.

There’s a lot of merit in using HTML/JS/CSS for large applications. I know of other companies that have used web tech for their tools, if you have a story to share, please do.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s