Part of The Lab that nobody has copied yet – drawing on a whiteboard


After the release of Valve’s The Lab we’ve seen huge amount of developers copying their mini-games and solutions. Endless number of archery games, teleporting systems with tiles and haptic feedback appeared over the course of last year. There is one part of The Lab that people seem to not notice: writing on a whiteboard with a marker. In the main scene of the Lab you can take a marker to your hand and draw some notes or paint some pictures on this whiteboard. I really liked this part and I was thinking that maybe lab developer that created this part might feel sad if he sees that everybody copies different part of their game but not his part. To not make him feel sad I decided to create simple mechanics in a blank Unity project using HTC Vive.

Algorithm

When I tried The Lab for the first time I noticed some “aliasing” artifacts on the whiteboard – I could identify single pixels on the canvas. My first guess was that there is a texture that covers entire whiteboard and the act of drawing is simply modifying pixels. There is a very simple technique that gives effect that looks exactly like that

Stamping

Stamping is a simple technique that allows you to draw with with one set of pixels over another using alpha blending. You can imagine it as a stamp texture moving over the given texture and writing to a result texture after performing alpha blend for each overlapping pixel.

The following formula was used to calculate the result colours and alpha values:

PaintReceiver.cs class is the meat of this project. It performs texture modifications and of course alpha blending. PaintOver is the method that performs the most important calculations:

If some parts of this method looks awkward to you – no worries. It is written like that due to performance improvements that are described later in this article.

Rotation

To achieve rotation of the stamp additional array was created – currentStampPixels. To rotate stamp we need to call RotateStamp(float angle) to set stamp rotation to given angle. I used a simple 2D rotation matrix:

Which means that:

To rotate texture around its centre this formula needs to be slightly modified:


Stamp.cs class holds information about alpha values for every pixel as an array of floats. Inside the Stamp class there is a public method:

CurrentPixels are the ones that are used to perform stamping.

Lines

First, I started to put one stamp in each Update call. This left me with some big gaps between stamps when I drew very fast. Due to that I decided to implement some sort of line generation using the stamp. In the final version, in every frame that the marker position has changed, the draw position is interpolated between the position from the last frame and the current one. Number of points between those two points can be adjusted by spacing values.
Spacing of 1 means that stamps are drawn right next to each other.

For example, if the drawing position from the last frame was (0,0), the position in the current frame is (30, 0) and my stamp size is (10,10) then my draw positions would be as follows:

(0, 0), (10, 0), (20, 0) and (30, 0)

Spacing of 0.5 means that the number of stamps that are drawn between these two points is twice as dense:

(0, 0), (5, 0), (10, 0), (15, 0), (20, 0), (25, 0), (30, 0)

Spacing of 0.25 means that this number is four times more dense.

Erasing

In The Lab we have an eraser. Implementation of that is fairly simple, the only thing we need to do is to is to add a check inside PaintOver method whether or not we are erasing instead of painting over. When erasing, we simply swap a stamp colour with original texture’s colour:

Optimisation

We are dealing here with a problem that is computationally very heavy. The complexity of calculations grows as a square of your stamp’s size, that’s why we need to perform some optimisations:

1. GetPixels32 and SetPixels32

When I started working on this project I used SetPixel and GetPixel. I noticed that using SetPixels32 and GetPixels32 combined with operating on bytes instead of floats gives much better results even though these methods operate on the entire texture. Since these methods operate on bytes I had to adjust my alpha blending method which in the C# code translates into:

2. Updating texture once per frame

It’s quite obvious, but you can perform all the pixel opperations within Update() method and then submit them once in LateUpdate() method when all the modifications are completed. You can do this by calling texture.Apply()

3. Replace redundant Mathf calls
When I tested my game with a profiler it turned out that calling Mathf.Clamp or Mathf.RountToInt several thousand times per frame really slows down my script and adds few milliseconds to the calculations. That’s why these inside PaintReceiver.cs were replaced with the following:

4. Don’t calculate anything for transparent stamp’s pixels
It’s simple as that, if calculations can be skipped for a particular pixels we should definitely do that! In this example I just chose an arbitrary small number to continue to the next pixel:

Project files

All the project files and full source code can be found here at my GitHub.

 


Thank you!!!

Share:

2 Comments

Add yours
  1. 2
    kamran

    Awesome job and thanks for sharing. In the lab version when you write the HTC controller has a nice vibration too which gives you feeling of realistic writing. I did not see it in you asset. Didn’t you include it or I have to activate somethin to aces it? Thanks again man of the year 😉

+ Leave a Comment