Saturday, January 2, 2016

But what if C++ was declarative to begin with?

New year, new career, time to come full circle with some old ideas...

It's a new year, and for the first time since 2003 I've accepted a job offer at a place that includes zero C++ programming.

Since this is the last I'll be using C++ or Qt in the near future (though, who knows...) I thought I'd share a thingie I've developed during the last few years.
It is not really a library or that much code... it is more of a style, an idea, an idiom, a drizzle of syntactic sugar.

This idea actually is quite simple so it may exist in the wild somewhere but I did try to google it and couldn't find something similar. So either I didn't Google the right keywords or it's a novel/bad idea.

What made me start thinking of this

It was a conversation while working on Qt at Nokia during the inception of QtQuick, back then "Declarative UI" and other names. I was a software engineer working with the Qt business development group, and remember a single quote from one of the other engineers that stayed with me: 
C++ is imperative.
 I had a response in my head at the time that I hadn't voiced because I couldn't quite actualize it yet:
But what if it was?
Time has passed and QtQuick grew to be a decent and legitimate set of tools for touch/device UI. However, it always had two built-in limitations (which don't take away from its strengths):

  • QML is not compiled, thus connecting it with other C++ code requires a lot of broilerplate shim code (QObjecty stuff)
  • QML is very tied to the Qt framework
I wanted to see how declarative programming can be used in C++, but inside the language and not with an additional interpreter which requires language bindings etc.

So what is declarative really?

Since conceiving and contributing to the Qt state machine framework together with the clever folks at Nokia/Trolltech, I have thought state machines were an interesting case for declarative programming. Taking them as a test case, I can define declarative programming as:
Setup of a hierarchical data structure that interfaces with the programmatic modules.
Hierarchical statecharts are a tree data structure, and it interfaces with the other modules - e.g. you want to change colors of a widget when a state is entered/exited.
So if we temporarily reduce "declarative" to defining "Tree-structured data that interfaces with code", it is clear why the old C++ could not easily be used for declarative programming. Defining tree data structures in the old C++ is very ugly (look at code that interfaces with the boost state machine frameworks for examples), and setting up a decent QStateMachine in C++ requires a lot of imperative "setThis" "setThat" lines.

Looking at what actually happens inside QML (take the rest of QtQuick aside for a sec), it converts a tree structure to a a set of commands to set properties. 

Delcaraitve is really about converting a data structure to a live tree with a setup phase.

Along came C++11

I was able to use C++11 in various projects I've been working on since mid 2013, when compiler support caught up with the particular product/platform roadmaps.
When I started looking into its details, between my two previous jobs, I found it had several key features that made me feel declarative programming was not just possible but potentially sweet:
  • Variadic templates
  • Move constructors
  • decltype
  • Lambdas

Looks a bit like QML, but not really

The idea was to take the nice bits about QML - mainly how the code looks and feels when constructing a tree data structure, and assimilate it into the C++ world.
In other words, build a curly-bracket tree representing hierarchical data inside C++.

The code for this, in C++, looks like this:

 Object1 {  
   Object1 { Flag1, Property1{0}, Property2 {"string"},  
     Object2 { }  
   },  
   Object2 { InlineFunction1 { [=] { doSomethingInLambda(); } }  
 }  

When described in words, what I do is this:
  • Use a variadic constructor as a way to declare a tree structure for an object (e.g. Object1)
  • Define special struct types (e.g. Property1) as a way to define properties
  • Define special enum types (e.g. Flag1) as a way to define flags.
To make this work, the different property, flag and object types need to be defined beforehand:

 // Define properties as structs  
 struct Property1 { int n; }  
 struct Property2 { std::string s; }  
 struct InlineFunction1 { std::function<void()> f; }  
 // Define flags as anonymous enums  
 enum { Flag1 };  

Then we define object types, like this:

 class Object1Base {  
 protected:  
   void declare(Flag1);  
   void declare(Property1);  
   void declare(Property2);  
   void declare(InlineFunction);  
   void declare(Object2);  
 };  
 class Object2Base {  
 public:  
 };  
 typedef Declarative<Object1Base> Object1;  
 typedef Declarative<Object2Base> Object2;  

The "Declarative" template class is the little magic (really rudimentary variadic args) that binds a declarative constructor like the example to the different "declare" functions.

The clarat code & back to QStates

To demonstrate how this ties back together, I've prepared a little header file that does the declarative constructor magic, and created, as an example, a binding to QStateMachine.
The code for a state machine, that would normally look like a bunch of setThis/setThat now looks like this:

     State {  
       State { Initial, Transition { Target { &test1 } } },  
       State { &test1,  
         State { Initial, Transition { Target { &test1Sub1 } } },  
         OnEntry { [=] { printf("Inside Test1\n"); } },  
         State { &test1Sub1,  
           OnEntry { [=] { printf("Inside Test1Sub1\n"); } },  
           OnExit { [=] { printf("Leaving Test1Sub1\n"); } },  
           Transition { On<Event1>(), Target { &test1Sub2 } }  
         },  
         Final { &test1Sub2 },  
         Transition { OnDone, Target { &test2 } }  
       },  
       State { &test2,  
         OnEntry { [=] { printf("Inside Test2\n"); qApp->exit(0); } }  
       }  
     }  

Isn't this much more manageable and tasty than writing a bunch of setter functions?
It was for me but maybe it's an acquired taste.

The full example is available freely at https://github.com/noamr/clarat. It works with the QStateMachine framework but the declarative aspect is not Qt-specific.



Summary


I've used code like this (not exactly this code, I've written this one now...) in production. It had helped me and the team maintain C++ UI/states setup code that usually becomes rather convoluted, and keep it clean and easily changeable.


I probably forgot to mention something extremely important, so please mention that one in the comments.
Since I'm doing this for free I can also promise twice your money back if any of you are not satisfied with what I'm sharing here :)
My biggest satisfaction would be from seeing this syntactic sugar being used out there in the world, making C++ setup code slightly nicer.

~Cheers
Noam

Saturday, September 13, 2014

A trivial way to work around boolean traps and bad APIs

Long time no blog...

In Ariya's blog from 2011, there were interesting ideas about good/bad APIs pertaining to booleans. To repeat, the main point, an API may be not so great if you can't understand the meaning of its parameters from the calling site.

For example (to recap), looking at an Xlib call site:

XClearArea(myDisplay, 0, 0, 100, 100, true);

What does "true" mean here? Since I'm not an X11 expert I have to go to the documentation to know that this means "whether or not to send expose events".

Today I don't have to use X11 much, but this is a problem in APIs I do have to use more often, like OpenGL:

glUniformMatrix4fv(matrixLocation, 1, GL_FALSE, matrix);

GL_FALSE here means that the matrix should not be transposed, but this is not clear from the call site.

Now there are many ways to tackle this in the design of the API itself, for example:

enum ShouldTransposeMatrix { DontTransposeMatrix, TransposeMatrix };

But... when using existing APIs we don't have the privilege to redesign the API.
Also, despite of it solving the problem in the call site, something about it feels too verbose... 3 new words just to represent a boolean parameter for a single function.

So...

In a project I've been working on we developed a coding style principle that makes this problem almost entirely go away, and in a trivial fashion.
We put this line somewhere on the top:
typedef GLboolean ShouldTransposeMatrix;  


... and then the calling site looks like:

glUniformMatrix4fv(matrixLocation, 1, ShouldTransposeMatrix(GL_FALSE), matrix);


Only for booleans?

Of course not! By using this "optional strong-typing" style any non-obvious parameter can look much clearer in the call site, without changing the API itself.

typedef GLboolean ShouldNormalize;
typedef GLsizei AttribStride;
typedef GLint AttribSize;
typedef GLenum AttribType;

glVertexAttribPointer(index, AttribSize(4), AttribType(GL_FLOAT), ShouldNormalize(GL_FALSE), AttribStride(4), nullptr);

This example is maybe going too far but it's a matter of taste... I like using this style mainly for places where the call site makes the API obviously non-obvious, and most often than not with booleans.

But isn't this trivial? Why should I care? Give me my two minutes of reading this blog back!
It is. You shouldn't. I can't.


Tuesday, March 8, 2011

Webkitten Hunting Tips

"So, do you know any available WebKit engineers?"
I've been hearing this phrase over and over ever since I put the word "WebKit" in my LinkedIn profile.
Since I'm tired of answering that question over and over again, and since I'm starting to feel sympathy towards the recruiters in their difficult rare-species scavenger hunt for webkittens, I thought I'd answer three times.

Short answer: no.
Slightly less short answer: All the "WebKit engineers" I know are connected to me on LinkedIn already, and you've already contacted them or are planning to.
Long answer: This is the wrong question. Continue if you want to know why.

Instead of searching for "WebKit engineers", that as you may have found out are few and far between, look for good C++ engineers. If you want to be more specific, find C++ engineers that worked on frameworks/reusable libraries or on cross platform code in general, since the kind of sensitivities you develop when developing reusable library code would be valuable in WebKit.
Another option for being specific, is to find engineers that work on open source projects. Their experience with working with the community and having their code constantly re-evaluated by others would be of help. If you want to be even more specific, you can combine the two (hire people with experience in open-source frameworks/libraries). For example, people who worked on Qt would feel pretty comfortable in the Webkit codebase.

I started working on WebKit around December 2009, became a committer about 4 months later, and feeling pretty comfortable around it by now. Back then I wasn't a "Webkit engineer", but I've become one because I was given the chance. Why not give this chance to others, if they're good at similar technologies or disciplines?

Q: This is all good, but what we're looking for are WebKit engineers. Do you know any?
A: No.

Happy hunting
No'am

Thursday, February 24, 2011

Antitecture

I'm experimenting with a new way of looking at software.
It's the wrong way of looking at software, but I'm experimenting with it anyway.
I call this antitecture - short for anti-architecture.
It's an ideology that claims that architecture is evil and is 100% a bad thing. I hope to make this into a cult and make followers pay me millions of dollars. but I digress.

The idea behind it is that architecture is about scale. In non-software architecture, you design a house on paper and then the house almost literally scales to its actual form. Fixing issues in a small scale later prevents having to solve those issues on a larger scale.
But a lot of what we do today in mobile user-experience doesn't need to scale. It's a bit of content and data that we need to present to the user in a nice way. Not difficult in itself, but gets difficult when we try to apply architecture and concepts to it.

So, if I write a music library that's divided into folders, I do just that - have a data structure of music divided into folders - no "metadata", no "tagging", no concepts. If later I'd need something more sophisticated, I'll do it later.

The bogus theory behind this ideology is that end users have simple needs, and thus the software that serves them should be simple. Our problems as developers begin when we try to apply concepts and architectures that we learned when we were in the "scalability" cult, which is maintained by the illuminati that controls the computer-science universities and the government, to a software discipline that doesn't require scalability, like some user-interface solutions for example.

Of course, some software does need to scale and that's where architecture is a good thing - but I was trying to make a point so don't confuse me with my fallacies.

Seriously though, I think my favorite way of looking at this is: architecture should be a function of scalability. The less your solution needs to be scalable, the less architecture it requires.

No'am