Creating RESTful applications with Qt and Cutelyst

This mini tutorial aims to show you the fundamentals of creating a RESTful application with Qt, as a client and as a server with the help of Cutelyst.

Services with REST APIs have become very popular in recent years, and interacting with them may be necessary to integrate with services and keep your application relevant, as well as it may be interesting to replace your own protocol with a REST implementation.

REST is very associated with JSON, however, JSON is not required for a service to become RESTful, the way data is exchanged is chosen by the one who defines the API, ie it is possible to have REST exchanging messages in XML or another format. We will use JSON for its popularity, simplicity and due to the QJsonDocument class being present in the Qt Core module.

A REST service is mainly characterized by making use of the little-used HTTP headers and methods, browsers basically use GET to get data and POST to send form and file data, however REST clients will use other methods like DELETE, PUT and HEAD, concerning headers many APIs define headers for authentication, for example X-Application-Token can contain a key generated only for the application of a user X, so that if this header does not contain the correct data it will not have access to the data.

Let’s start by defining the server API:

  • /api/v1/users
    • GET – Gets the list of users
      • Answer: [“uuid1”, “uuid2”]
    • POST – Register new user
      • Send: {“name”: “someone”, “age”: 32}
      • Answer: {“status”: “ok / error”, “uuid”: “new user uuid”, “error”: “msg in case of error”}
  • /api/v1/users/ – where UUID should be replaced by the user’s UUID
    • GET – Gets user information
      • Answer: {“name”: “someone”, “age”: 32}
    • PUT – Update user information
      • Send: {“name”: “someone”, “age”: 57}
      • Answer: {“status”: “ok / error”, “error”: “msg in case of error”}
    • DELETE – Delete user
      • Answer: {“status”: “ok / error”, “error”: “msg in case of error”}

For the sake of simplicity we will store the data using QSettings, we do not recommend it for real applications, but Sql or something like that escapes from the scope of this tutorial. We also assume that you already have Qt and Cutelyst installed, the code is available at https://github.com/ceciletti/example-qt-cutelystrest

Part 1 – RESTful Server with C ++, Cutelyst and Qt

First we create the server application:

$ cutelyst2 --create-app ServerREST

And then we will create the Controller that will have the API methods:

$ cutelyst2 --controller ApiV1

Once the new class has been instantiated in serverrest.cpp, init() method with:

#include "apiv1.h"

bool ServerREST::init()
{
    new ApiV1 (this);
    ...

Add the following methods to the file “apiv1.h”

C_ATTR(users, :Local :AutoArgs :ActionClass(REST))
void users(Context *c);

C_ATTR(users_GET, :Private)
void users_GET(Context *c);

C_ATTR(users_POST, :Private)
void users_POST(Context *c);

C_ATTR(users_uuid, :Path('users') :AutoArgs :ActionClass(REST))
void users_uuid(Context *c, const QString &uuid);

C_ATTR(users_uuid_GET, :Private)
void users_uuid_GET(Context *c, const QString &uuid);

C_ATTR(users_uuid_PUT, :Private)
void users_uuid_PUT(Context *c, const QString &uuid);

C_ATTR(users_uuid_DELETE, :Private)
void users_uuid_DELETE(Context *c, const QString &uuid);

The C_ATTR macro is used to add metadata about the class that the MOC will generate, so Cutelyst knows how to map the URLs to those functions.

  • :Local – Map method name to URL by generating /api/v1/users
  • :AutoArgs – Automatically checks the number of arguments after the Context *, in users_uuid we have only one, so the method will be called if the URL is /api/v1/users/any-thing
  • :ActionClass(REST) ​​- Will load the REST plugin that will create an Action class to take care of this method, ActionREST will call the other methods depending on the called method
  • :Private – Registers the action as private in Cutelyst, so that it is not directly accessible via URL

This is enough to have an automatic mapping depending on the HTTP method for each function, it is important to note that the first function (without _METHOD) is always executed, for more information see the API of ActionREST

For brevity I will show only the GET code of users, the rest can be seen in GitHub:

void ApiV1::users_GET(Context *c)
{
    QSettings s;
    const QStringList uuids = s.childGroups();

    c->response()->setJsonArrayBody(QJsonArray::fromStringList(uuids));
}

After all the implemented methods start the server:

cutelyst2 -r --server --app-file path_to_it

To test the API you can test a POST with curl:

curl -H "Content-Type: application/json" -X POST -d '{"name": "someone", "age": 32}' http://localhost:3000/api/v1/users

Okay, now you have a REST server application, made with Qt, with one of the fastest answers in the old west 🙂

No, it’s serious, check out the benchmarks.

Now let’s go to part 2, which is to create the client application that will consume this API.

Part 2 – REST Client Application

First create a QWidgets project with a QMainWindow, the goal here is just to see how to create REST requests from Qt code, so we assume that you are already familiar with creating graphical interfaces with it.

Our interface will be composed of:

  • 1 – QComboBox where we will list users’ UUIDs
  • 1 – QLineEdit to enter and display the user name
  • 1 – QSpinBox to enter and view user age
  • 2 – QPushButton
    • To create or update a user’s registry
    • To delete the user record

Once designed the interface, our QMainWindow sub-class needs to have a pointer to QNetworkAccessManager, this is the class responsible for handling communication with network services such as HTTP and FTP. This class works asynchronously, it has the same operation as a browser that will create up to 6 simultaneous connections to the same server, if you have made more requests at the same time it will put them in a queue (or pipeline them if set).

Then create a QNetworkAccessManager *m_nam; as a member of your class so we can reuse it. Our request to obtain the list of users will be quite simple:

QNetworkRequest request(QUrl("http://localhost:3000/api/v1/users"));

QNetworkReply *reply = m_nam->get(request);
connect(reply, &QNetworkReply::finished, this, [this, reply] {
    reply->deleteLater();
    const QJsonDocument doc = QJsonDocument::fromJson(reply->readAll());
    const QJsonArray array = doc.array();

    for (const QJsonValue &value : array) {
        ui->uuidCB->addItem(value.toString());
    }
});

This fills with the data via GET from the server our QComboBox, now we will see the registration code which is a little more complex:

QNetworkRequest request(QUrl("http://localhost:3000/api/v1/users"));
request.setHeader(QNetworkRequest::ContentTypeHeader, "application/json");

QJsonObject obj {
    {"name", ui->nameLE->text()},
    ("age", ui->ageSP->value()}
};

QNetworkReply *reply = m_nam->post(request, QJsonDocument(obj).toJson());
connect(reply, &QNetworkReply::finished, this, [this, reply] {
    reply->deleteLater();
    const QJsonDocument doc = QJsonDocument::fromJson(reply->readAll());
    const QJsonObject obj = doc.object();

    if (obj.value("status").toString() == "ok") {
        ui->uuidCB->addItem(obj.value("uuid").toString());
    } else {
        qWarning() << "ERROR" << obj.value("error").toString();
    }
});

With the above code we send an HTTP request using the POST method, like PUT it accepts sending data to the server. It is important to inform the server with what kind of data it will be dealing with, so the “Content-Type” header is set to “application/json”, Qt issues a warning on the terminal if the content type has not been defined. As soon as the server responds we add the new UUID in the combobox so that it stays up to date without having to get all UUIDs again.

As demonstrated QNetworkAccessManager already has methods ready for the most common REST actions, however if you wanted to send a request of type OPTIONS for example will have to create a request of type CustomOperation:

m_nam->sendCustomRequest("OPTIONS", request);

Did you like the article? Help by giving a star to Cutelyst and/or supporting me on Patreon

https://github.com/cutelyst/cutelyst

Advertisements
Creating RESTful applications with Qt and Cutelyst

Cutelyst 2.3.0 released

Cutelyst – The C++ Web Framework built with Qt, has a new release.

In this release a behavior change was made, when asking for POST or URL query parameters and cookies that have multiple keys the last inserted one (closer to the right) is returned, previously QMap was filled in reverse order so that values() would have them in left to right order. However this is not desired and most other frameworks also return the last inserted value. To still have the ordered list Request::queryParameters(“key”) builds a list in the left to right order (while QMap::values() will have them reversed).

Some fixes on FastCGI implementation as well as properly getting values when uWSGI FastCGI protocol was in use.

Lastly a crash when performing TechEmpower benchmarks resulted in the removal of some nested event loops calls that were creating a huge stack, and made me realize that my proposed methods to make async apps easier were broken by design, basically you can’t use QEventLoop::exec() to wait for events. So Context::wait() and next() are deprecated and the code disabled, with the exception of WebSockets until I can make some changes to HTTP/1 and FastCGI parser both are to be considered synchronous.

Removing a QCoreApplication::processEvents() that was added in order to try to give other connections an execution slice almost doubled the speed in plain text when benchmarking locally, using a single core the requests per second have gone from 105 K to 190 K, after understanding the stupidity I did this isn’t of much surprise 🙂

https://github.com/cutelyst/cutelyst/releases/tag/v2.3.0

Cutelyst 2.3.0 released

Virtlyst 1.1.0 released

Virtlyst – a libvirt web interface to manage virtual machines has a new release.

This release finishes support to connect to TCP or TLS virtd servers, it also fixes creating new instances from the flavor panel. And a few other fixes.

Of course I forgot to add the mandatory screenshot last time so here it goes:

instances

console

Go get it! https://github.com/cutelyst/Virtlyst/releases/tag/v1.1.0

If you like this software you can also support my work via patreon.

Virtlyst 1.1.0 released

Cutelyst 2.2.0 is out

Cutelyst just got a new release!

Thanks to the release of Virtlyst – a web manager for virtual machines, several bugs were found and fixed in Cutelyst, the most important one in this release is the WebSockets implementation that broke due the addition of HTTP/2 to cutelyst-wsgi2.

Fixing this was quite interesting as when I fixed the issue the first time, it started to make deleteLater() calls on null objects which didn’t crash but since Qt warn’s you about that it was hurting performance. Then I fixed the deleteLater() calls and WebSockets broke again 🙂 a little of gdb foo and I found out that when I asked for the websocket to close Qt was emitting disconnected as a result of that call, and that signal was deleting the object we rely when we called close, which crashed the app.

As a new feature Context class has two new methods to help with async apps, a wait() method that creates a counter based local event loop and a next() method to decrease the event loop counter till quitting it. This way you can for example connect two QNetworkReply::finished() signals to Context::next() and call Context::wait(2); so once the two requests are finished wait() returns and the client receives the data.

Codacy was also added to increase the number of checks in our code base, and it has already shown some real issues.

UPDATE: Right after release I found a cppcheck code fix broke listening on TCP sockets, yes, writing more unit tests especially for the WSGI module is top on my TODO list. So 2.2.1 is out now 🙂

Get it here https://github.com/cutelyst/cutelyst/releases/tag/v2.2.1

Cutelyst 2.2.0 is out

Announcing Virtlyst – a web interface to manage virtual machines

Virtlyst is a web tool that allows you to manage virtual machines.

In essence it’s a clone of webvirtmgr, but using Cutelyst as the backend, the reasoning behind this was that my father in law needs a server for his ASP app on a Win2k server, the server has only 4 GiB of RAM and after a week running webvirtmgr it was eating 300 MiB close to 10% of all available RAM. To get a VNC or SPICE tunnel it spawns websockify which on each new instance around 20 MiB of RAM get’s used.

I found this unacceptable, a tool that is only going to be used once in a while, like if the win2k freezes or goes BSOD, CPU usage while higher didn’t play a role on this.

Choosing a web interface for KVM/libvirt is no easy task, I have used Archipel and while it’s a nice app it is a pain to install, OpenStack is also overly complicated, and had a button labeled as “Terminate” button that deleted the instance entirely. A simple tool that’s specially simple to install was what I needed, and in fact is what a bunch of people need, here at work, OpenStack was also discarded due the hard installation process and a mix of virsh/virt-manager is used.

Since webvirtmgr is a DJango app, using it’s html templates was a breeze, Grantlee didn’t work with them out of the box, it has a few missing features but one can work around those, libvirt API is also quite well documented so development was quite fast, it took me 3 weeks but could easily be reduced to less than 2 if I had a bit more time.

So Virtlyst v1.0.0 is out, it uses less than 10MiB of RAM, and implements the VNC/SPICE web socket proxy in the same process thus each connections increases a almost nothing of memory usage. The software is already in production, and has all features of webvirtmgr except the migration part, that I plan to add in a future release.

Following the steps of some developer fellows I also created a Patreon account, so if you like what I do and would like to support my work please consider becoming a Patron.

Enough said, go get it!

https://github.com/cutelyst/Virtlyst/archive/v1.0.0.tar.gz

Announcing Virtlyst – a web interface to manage virtual machines

Cutelyst 2.1 Released

Cutelyst a Qt/C++ Web Framework got upped to 2.1. Yesterday I found a bug that although not critical it’s important enough to roll out a new release.

As a new feature we have LangSelect plugin to help with auto detection of user language (Matthias).

And some important bug fixes:

  • Fix compilation with Qt 5.11
  • clazy fixes
  • Fix undefined cutelyst2_plugin symbol of uWSGI plugin
  • Fix for IPv6 address binding
  • Fix StoreHtpasswd::addUser if file doesn’t exist (Bart)
  • Fix parsing when the url encoded (on body or query) has a key without a value but carries the equal sign (?foo=&bar)

Get it https://github.com/cutelyst/cutelyst/archive/v2.1.0.tar.gz

 

Cutelyst 2.1 Released

Cutelyst 2 released with HTTP/2 support

Cutelyst the Qt/C++ web framework just got a major release update, around one and half year ago Cutelyst v1 got the first release with a stable API/ABI, many improvements where made during this period but now it was time to clean up the mistakes and give room for new features.

Porting applications to v2 is mostly a breeze, since most API changes were done on the Engine class replacing 1 with 2 and recompiling should be enough on most cases, at least this was the case for CMlyst, Stickyst and my personal applications.

Due cleanup Cutelyst Core module got a size reduction, and WSGI module increased a bit due the new HTTP/2 parser. Windows MSVC was finally able to build and test all modules.

WSGI module now defaults to using our custom EPoll event loop (can be switched back to Qt’s default one with an environment variable), this allows for a steady performance without degradation when an increased number of simultaneous connections is made.

Validators plugins by Matthias got their share of improvements and a new password quality validator was added, plus manual pages for the tools.

The HTTP/2 parser adds more value to our framework, it’s binary nature makes it very easy to implement, in two days most of it was already working but HTTP/2 comes with a dependency, called HPACK which has it’s own RFC. HPACK is the header compression mechanism created for HTTP/2 because gzip compression as used in SPDY had security issues when on HTTPS called CRIME .

The problem is that HPACK is not very trivial to implement and it took many hours and made KCalc my best friend when converting hex to binary to decimal and what not…

Cutelyst HTTP/2 parser passes all tests of a tool named h2spec, using h2load it even showed more requests per second than HTTP/1 but it’s complicated to benchmark this two different protocols specially with different load tools.

Upgrading from HTTP/1.1 is supported with a switch, as well as enabling H2 on HTTPS using the ALPN negotiation (which is the only option browsers support), H2C or HTTP/2 in clear text is also supported but it’s only useful if the client can connect with previous knowledge.

If you know HTTP/2 your question is: “Does it support server push?”. No it doesn’t at the moment, SERVER_PUSH is a feature that allows the server to send CSS, Javascript without the browser asking for it, so it can avoid the request the browser would do, however this feature isn’t magical, it won’t make slow websites super fast , it’s also hard to do right, and each browser has it’s own complicated issues with this feature.

I strongly recommend reading this https://jakearchibald.com/2017/h2-push-tougher-than-i-thought/ .

This does not mean SERVER_PUSH won’t be implemented, quite the opposite, due the need to implement it properly I want more time to study the RFC and browsers behavior so that I can provide a good API.

I have also done some last minute performance improvements with the help of KDAB Hotspot/perf, and I must say that the days of profiling with weird/huge perf command line options are gone, awesome tool!

Get it! https://github.com/cutelyst/cutelyst/archive/v2.0.0.tar.gz

If you like it please give us a star on GitHub!

Have fun!

Cutelyst 2 released with HTTP/2 support