Building a Dynamic UI System at Carousell (Part 3)

Posted on:March 5, 2019

This is the third and final part in a trilogy of articles about a dynamic UI system we’ve developed at Carousell over the past one and a half years to solve a series of related problems.

In the first part, we talked about some problems we faced at Carousell, and in the second part, we introduced fieldsets and how the system works by showing some examples of client-server interactions. This third part will talk an assortment of topics related to the Fieldset system.

These are some of the topics we will touch on in this part:

Were fieldsets inspired from other similar systems? What did the prior art look like?

Back during the early planning phase of fieldsets in April 2017, we were unable to find much literature on developing such systems, and none of us in the team had experience building anything like it. So at that time, the blank slate was both challenging and daunting. Fieldsets was born out of not just what we needed at then, but sheer imagination of how we could reasonably predict Carousell evolving in the next few years.

Since then, Airbnb has published something which seems to be similar in spirit.

How do we manage (the large number of) fieldsets?

Because we couldn’t predict how fieldsets would be used, we decided to go with the easiest and most straightforward approach that we could think of, and evolve as needed. The approach was to keep the fieldset definitions under version control with Git. While Git provides a nice GUI1 and version control, it doesn’t provide anything in the way of fieldset-specific features. Something we’ve since observed being done a lot, is mass modifications of fieldsets, like updating a component being shared by many fieldsets with a new field (see the section on fieldset markup below).

We’re working on an internal tool to make these kinds of changes easier by allowing certain classes of mass modifications trivial, while exposing an escape hatch for scripting more complex changes.

How we decide the fieldset markup for a component

The markup for a component refers to the actual definition of the component. For a text component, it might look like this:

{
  "id": "field_0",
  "meta": {
    "component": "text",
    "field_name": "title"
  },
  "ui_rules": {
    "keyboard_type": "text",
    "label": "Title",
    "placeholder": "Enter a title for the listing",
    "visible": true
  },
  "validation_rules": [
    /*snip*/
  ]
}

One can think of this markup as the “latitude” of the component — that is, how much flexibility a component has. Flexibility of a component is determined by a few guiding questions:

If a component is something that’s cut-and-dried, then there’s little point in declaring every property of the component. Instead, it should be left up to the client’s fieldset rendering system to decide its look-and-feel.

On the other hand, the more complex and flexible a component needs to be, the more knobs and buttons we’ll have to expose in the component’s markup. For example, if it was decided that font colour of the placeholder text has an effect on how likely it is for users to fill the field up, we’ll expose it like that:

{
  "id": "field_0",
  "meta": {
    "component": "text",
    "field_name": "title"
  },
  "ui_rules": {
    "keyboard_type": "text",
    "label": "Title",
    "placeholder": "Enter a title for the listing",
    "placeholder_colour": "#ffffff",
    "visible": true
  },
  "validation_rules": [
    /*snip*/
  ]
}

Of course, this is more of an art than a science, and many teams have designed components with varying styles of markup, which is why we’re currently working on a set of guidelines and examples for designing new components.

This is also where fieldsets have a strong tie-in with a design system, something that the Product Design team at Carousell is working hard on developing.

How the development and deployment process looks like

Defining a new fieldset typically involves translating feature requirements into components. Once this is done, new components are worked on (mostly on the client side, since it’s just adding some markup for the new component on the backend). The new fieldset is checked into the Git repository, and tested in a staging environment before being released into production.

The deployment pipeline currently involves a Jenkins pipeline that reads the fieldsets from a Git repository, runs the unit test suite, and then uploads the validated fieldsets to Fieldset service.

How the testing suites work and what the deployment pipeline looks like

Testing for fieldsets is a tricky business. We have a Python script that generates unit test cases to check all fieldsets and make sure that every component’s markup is correct. It works well in catching various classes of errors but it can be clunky to manage, especially as more and more new components get added.

At the time of writing, we’re almost done with the process of porting component definitions and validations into Go in the form of structs and interfaces. There are two main reasons why we’re doing this. It allows fieldsets to be much more easily and safely handled on the server side2, and it also serves as some level of documentation for components.

How we do documentation

For any given component, the most common questions are:

We do not yet have documentation that addresses all of these questions in a way that’s easy to peruse. The best source of information for now remains to be the fieldsets that are currently in production, but wading through a sea of JSON is not something that’s efficient. This is something that we’ll definitely be working on in the near future.

How we do localization

Localizable strings are stored in fieldsets are special tag strings, which have the following syntax TAG_SOMETHING. For example:

{
  "id": "field_0",
  "meta": {
    "component": "paragraph",
    "value": "Nice T-shirt",
    "field_name": "title"
  },
  "ui_rules": {
    "label": "TAG_TITLE",
    "visible": true
  }
}

Before fieldsets are served to clients, it goes through a localization layer, which converts these tag strings to their localized versions:

{
  "id": "field_0",
  "meta": {
    "component": "paragraph",
    "value": "Nice T-shirt",
    "field_name": "title"
  },
  "ui_rules": {
    "label": "標題",
    "visible": true
  }
}

(Protip: Go has an excellent golang.org/x/text/language library that handles the heavy lifting of parsing the Accept-Language header and performing language tag canonicalization.)

I hope this series of articles has given you a glimpse into the UI system that powers a large part of Carousell. Feel free to ask any questions!

Footnotes

  1. By way of Git{hub,lab}. ↩

  2. Fieldsets were treated as raw JSON previously, and we had to write a lot of defensive checks to make sure things were running smoothly. ↩