CMlyst 0.3.0 released

CMlyst is a Web Content Management System built using Cutelyst, it was initially inspired by WordPress and then Ghost. So it’s a mixture of both.

Two years ago I did it’s first release, and since them I’ve been slowly improving it, it’s been on production for that long providing web site/blog. The 0.2.0 release was a silent one which marks the transition from QSettings storage to sqlite.

Storing content on QSettings is at first quite interesting since it’s easy to use but it showed not suitable very fast, first it kept leaving .lock files, then it’s not very fast to access so I had used a cache with all data, and a notifier updated that when something changed on the directory, but this also didn’t properly triggered QFileSystemWatcher so once a new page was out the cache wasn’t properly updated.

Once it was ported to sqlite, I decided to study how Ghost worked, this was mainly due many Qt/KDE developer switching to it. Ghost is quite simplistic, so it was very easy to try to provide something quite compatible with it, porting a Ghost theme to CMlyst requires very little changes due it’s syntax being close to Grantlee/Django.

Due porting to sqlite it also became clear that an export/import tool was needed, so you can now import/export it in JSON format, pretty close to Ghost, actually you can even import all you Ghost pages with it, but the opposite won’t work, and that’s because we store pages as HTML not Markdown, my feeling about markdown is that it is simple to use, convenient to geeks but it’s yet another thing to teach users which can simply use a WYSIWYG editor.

Security wise you need to be sure that both Markdown and HTML are safe, and CMlyst doesn’t do this, so if you put it on production be sure that only users that know what they are doing use it, you can even break the layout with a not closed tag.

But don’t worry, I’m working on a fix for this, html-qt is a WHATWG HTML5 specification parser, mostly complete, but the part to have a DOM, is not done yet, with it, I can make sure the HTML won’t break layout and remove unsafe tags.

Feature wise, CMlyst has 80% of Ghost features, if you like it please help add missing features to Admin page.

Some cool numbers

Comparing CMlyst to Ghost can be trick, but it’s interesting to see the numbers.

Memory usage:

  • CMlyst uses ~5MB
  • Ghost uses ~120MB

Requests per second (using the same page content)

  • CMlyst 3500/rps (production mode), 1108/rps (developer mode)
  • Ghost 100/rps (production mode)

While the RPS number is very different, on production you can use NGINX cache which would make the slow Ghost RPS not a problem, but that comes to a price of more storage and RAM usage, if you run on an AWS micro instance with 1GB of RAM this means you can have a lot less instances running at the same time, some simple math shows you could have 200 CMlyst instaces vs 8 of Ghost.

Try it!

Sadly it’s also liked soon I’ll be forking Grantlee, the lack of maintenance just hit me yesterday (when I was going to release this), Qt 5.7+ has changed QDateTime::toString() to include TZ data which broke Grantlee date filter which isn’t expecting that, so I had to do a weird workaround marking date as local to avoid the extra information.

CMlyst 0.3.0 released

13 thoughts on “CMlyst 0.3.0 released

    1. dantti says:

      I have done that, but look at PR there’s stuff from 2015, I asked to become a maintainer but got no reply

      1. Well I understand your frustration because I’ve been in the same place countless times, but I just checked and of the two pull requests I can see under your name, in the first one you were asked to split the commit into multiple ones and in the other one the maintainer asked for some clarification and got no answer. On the issues side, you have 3 without response, so I guess the guy is pretty busy, although he at least checks pull requests.

        It is clear the maintainer is picky and strives for perfection, which has ennerved me multiple times in other projects I tried to contribute to before, but in the long run it’s a good thing even if it might set back some contributions like in your case. Just try harder before forking 😉

    1. dantti says:

      Well he owns the DNS, git in his personal github, he’s a great programmer, but too busy. And it’s an important add-on of Cutelyst so I need faster response. All my previous contributions took a lot of time to get merged in…

    1. dantti says:

      On the first server run set a SETUP enviroment variable so that you can register an admin user and password, you then unset it and is good to go.

      1. Flesh says:

        Sorry was trying to do this:
        that did not work, this did:
        declare -x SETUP=true
        I should have remembered that it needs to be exported, it would be nice to say what command to run to set it for people like me who forgot to export it, I guess I could have used the export SETUP, not sure what is more portable with Bash, or other Terminals like sh.
        I will be checking this project out, I wrote a CMS in Wt and was never happy with it, I could not manipulate the Head file, I need to put metadata and change themes in the header so the css would load before the page, and not after it, which is what Wt does after if loads the original page, it used ajax, and never updated the header after the first load, hoping this one does not do that.

        Do you have any Documentation?

        Thanks for getting back so fast, I really like the concept and work I see so far, and now I am trying to learn how to use it, the tutorial you did for CMlyst was great, maybe one for this project would be great.

        If I like it I will help out with adding admin options and documentation, and multilingual support.

        My interest is in using this to write Books I can output in ePub, eBook formats like Mobi for Kindle, and PDF, as well as HTML online Book, with a Print layout that works, so on the server it has to run epubchecker, kinglegen, and wkhtmltopdf; but my main goal is to make it multilingual, so all its text is based on language, so it separates the HTML from the text, and will need a way to do computer translations in the back end, so the app will focus on not only blogs, vlogs, and other types of web content, but also books that can also be outputted as Scripts or Screen Plays, so it handles text in context to its output, so you can mix a screen play, script and book all in one file, yet output all three or more, while still making it a web page or web book, or ebook in html, because a Website can also be laid out like a book, it has a table of Content or Menu System.

        It needs to work on Mobil devices as an App, as well as a Webserver for web pages, so it takes CMS to the next level, publishing Books, eBooks, Printed Books, Scripts, Screen Plays, and other Project Manager features to help with projection and management of documents, be it: Public, Private or paid content, the type of system I want to build must be flexible enough to handle anything.

        By the way, can you publish a pro file for QtCreator? I could not find one, and I read somewhere that you use it and like it, as I do as well, I created one, just have not put in all its library dependencies yet, but it would be nice to run the same one you do.

      2. dantti says:

        Sorry, didn’t get that you didn’t understand what the SETUP mean. On cutelyst 1.8 I’m likely to add –env option like uwsgi has, so SETUP could be set there.
        As for translations, once html-qt is finished getting text to a file like .po, and adding some kind of translation interface wouldn’t be hard, but my priority to call CMlyst production ready is to get html-qt produce safe HTML.
        The way themming works is like Ghost, you can look at the default theme, the template tags are pretty close to Ghost so you can take a look at it’s docs, sadly I don’t have much time to add docs for this ATM.
        I don’t use pro files, this project is CMake based I like it much more, I just tell QtCreator to open CMakeLists.txt.

  1. Flesh says:

    qEnvironmentVariableIsSet was added and I just never used it before, learn something new everyday, as I did not know I could open CMakeLists.txt, that is very cool, been doing it the hard way all these years, and I stared with it when it was called troll, I really like it, its getting better all the time. I ran into an issues with the database and name, I filled an issue, the 4th issue, anyway, I am interested in learning about it and I did read about Ghost, a lot to learn.

    Have you given any thought as to how to Document?
    Doxygen is ok, years ago I started a project and I used a class to write all documentation to an HTML file, the code was put in a separate file and only included in a helper app, it had built in help and the ability to make a new help file from a function in every class that documented every function in a class, it used .po files to store all the info in, so you could pick what language or it would automatically detect it, I thought it was better then Doxygen, which did not account for multilingual comments, but it also required that you support all the languages you offer, or have some type of translation app to do the work for you, but then it makes it hard to do multilingual comments, since that strategy normally deletes or overwrites all language files, its hard to keep them in sync once you start a project, so its best to write the app to delete all .po files and build them from new, this requires an array with each element being a language, the programmers can comment in any array for that language, then the program will only replace empty fields from computer translations, which I normally did it all by hand, mostly because I never found a service that works all that great at doing translations, but its clear this concept of self documenting is not normal as far as I know, people I have tired to explain it too, though it was too difficult to implement, which its only one multidimensional array; it has the name of the function, parameters pass to it, return codes, notes, author, date created, date last updated, version, I wrote it so it defaults all blank fields as it loops through it, so if you only fill out the dates, author and version, it will not change, it also has another field for history, this has user id’s, and reason for changes, this can be used to update history changes and what is new pages, bug fixes, and it can be used to track bug fixes, for example, you have a simple multidimensional array in json format, each line is a record, each line has a language id, like en for English, so on, so if someone reads it needs it to be in another language, they just copy the line, change the language field, and translate the words in the fields they need, the others they leave blank will be carried over from other lines that are not blank, each record has an ID with is just a sequential record number and category, for example: function_name, arguments, return, help, notes, history, author, coauthor, editor, qa, bug, and a number of categories you just can not live without, so you keep this array in a file with the same name with a -helper at the end of it, its a .h file, and only included in the helper app, so it adds no weight to it like Doxygen does, and it gets messy with multilingual comments, at least it did last time I used it, which was a long time ago, maybe it handles it better nowadays, but I think its still just commented code above a function, which adds dead weight to a program, not complied weight mind you, so it makes no difference to the size of the library, but to me its just to hard to deal with if you want to work with people that do not speak the same language, we still write in it, which is really strange when you think about it, we are not from the same country, yet we can work on the same code, but wouldn’t it be nice to use our Native language to Document things in? This approach can, and its concept works great, I use it in Bash Scripts, the one that I want to use this app to replace in fact, so think about it, this is a huge step to make, its cutting edge for self documenting and self multilingual CMS, you do not even have to finish the helper app first to use it, just figure out what json format to use, then send it to a template to print it to an HTML document you can convert to any format like PDF, ePub or Mobi for starts, you just need a helper app so you can sort by category, so you can look at just the Help Category or Bug Category, so you need a field for status like: fixed, not-confirmed, confirmed, …, as long as you have a good json file that fully describes everything you need it to, its easy to write a helper around it, and then people like me can just open it and look at it, since its a fully readable file without a program to format it, its just XML in UTF-8, so it can contain every language you support, which are the ones that the users are adding, no reason to add blank fields, only those that actually have content that does not need to be translated, and its record needs to be tied to the one it translates, so it needs what I call an MID, or Master ID, so what ever ID the field you copy becomes the MID and then you replace the NEW ID with a unique number, and if you copy a record and put it at the end, its always going to be sequential, its only if you try to insert it between existing records that you would need to renumber the list, its not a real database, but it can be written to one, but this is all done by hand, but you could just write an App that does this and make life simpler for everyone, call it the Documentor, use your own CMS to self document itself, write comments above and before each function you document, this can be done by the app after you chose to add a comment to it, you pick what type or category to add, fill in all the data required for that type, and then mark the line, so the programmers know the record number to look for, and they know by looking at the code that its been documented, and if you add categories to the comment, they will know what type of data the helper has, for example it has open bug reports, so its like flagging code with a FIXME, its easy to search for. Writing the app will give you the ability to write it to a file such as in json in a .h file, so they can just open it and do not have to pull up a web page to view it, or just keep it in a database, but it needs to be flexible, and this concept is, and its easy to implement.

    Also your Database type can not be hard coded, it needs to be able to use what ever Database the User wants, not just SQLite, easy to do, but I got the feeling by reading about converting this to SQlite that its hard coded, and from what I have seen of the code, it does not need to be, just make a define statement for every database you will support, like: PostGree, MySQL, and others that are easy to use, grant you this has to done using DEFINES and its compiled in, I have tried that on the fly idea, its adding databases calls to databases you may not even have installed, so it should be hard coded for the type defined, and then all the code can be type checked at compile time, and you can abstract the data types using a library that already handles this type of CRUD, I know you have written about this, but I don’t see it being implemented here, but this is just my first day looking at the code, and I have a million questions, but really I can learn this over time and do not need them answer right now, but it all comes down to how can I help, and writing the help system might help, but first we have to get the program running, so unless I am doing something wrong, there is an issues with name, but like I said, without Documentation its hard to figure all this out, which is why one of the categories needs to be work flow, so you can follow it from call to call and know the flow of function, so you can write flow charts, also you need self test built in, I like doing this with files with the same name with a -test at the end. it can be example code, but it gives the programmers a way to test and show examples of how to use functions, for example, I would write a test to addUser, then I would know if its going to work for fail, as before, I only link this code to the helper app.

    You can also track benchmarks using this method, document call times and bottle necks, but in method calls to use alternate code, so you can test different ways to do things and keep track of execution time, so you know by the history if its getting faster, you can also write a test to do just that, and use this information to write a benchmark profile for the App itself.

    So this is something to think about, it would only require a few extra files to be created for each or you can just combine them into one file and name it the same as the file with -loctest, or something, just so its clear that this is not a normal file, or put them in a separate folder, that way its a cleaner file structure, and you can use .cpp or .h, but you need to execute code, and you can do that in a .h, so its really just a matter of standardizing, and no one I know is using a system like this, they are all using tools that have been around for a long time, there are test suits and other tools that exist today, but over the years, I never really liked any of them, as a computer programmer, I like to work in code, not document managers, or Doxygen code, or having to write automated test code, when a better test could be written by hand, but lets face it, anything is better then nothing, and no documents sucks for everyone, and figuring out what method you want to document is a huge deal to any project, this one can make it different, it can not break it, and it will work, so its just a matter of if you want to use it over what is out there, so ask yourself how you want the Multilingual side of this app to work, and if you think about it, its not that much different from documenting it, its all about working with .po files, which means you need to have a record for every language in your database for every record you have for lets say a post, sure if its blank you can just create a record from a computer translator, but letting Google Translate to translate your website is not a good option, and not an option in every country, and my goal is to publish ePub, eBooks, Printed Books, PDF, and Web Pages all from the same source, this gives me the option to release my Website as a Book and not just as an online website, and those need to be translated for me, and that is what this system is written to do do, but it has to be integrated into the whole system, so it takes into account what language the user is writing in, so you can add and update the correct record, so you might have a post that was written in one language and then translated into another by hand, what happens to the original, and how do you know what is translated by hand, and what was done by machine, so you do not overwrite hand translations, but also data that is updated may not get updated by the person who translated the old data, so you have two different versions of the same data, one edited and one not, so you have to have a way to manage this in the back end, and it will be hard if you wait til the end to add this functionality, but for the users, this is something they really need, and the programmers as well, its two different programs, the back-end for the main app and the helper function that will run on another port, so you can password protect it, or allow only specific IP’s access to it, which is actually more secure, I always require it to be keyed for ssh access only, and security is a major issue, but from the stand point of the end user, it just needs to work, and from the programmers side, it must do everything we need it too and more, it becomes a project in itself, and a concept that others will want to use for their projects, a self testing self documenting app.

    You start out writing a CMS because you do not like the others, you have a great idea, this is what I was hopping that Wt would do, the theme idea is great, I have not worked with it yet, but it sounds great, templates also look great, I would like to add more feature to them, also what about Plugins, now your head starts to spin, I want to localize, and I want to document everything, so the list grows, as does the concept of what a CMS should do, so think about all the aspects that need to be dealt with upfront: Localization, Documentation, Security, Database Support, Device Support, output support, PDF is almost a given, but ePub and Mobi that some serious work, and I know how to do this, or I am learning as I write a book on it, its opened my eyes to what a Real CMS needs to do, which is Manage my Content, and you want it to be Fast, real fast, the Fastest in fact, and the most Secure, and it has to be Multilingual, what is this what I started off writing or is this a totally new CMS?

  2. Do you have a Forum or Email to discus the CMS?
    I am trying to learn how to use it, but running into problems, for one, I installed this on Linux (Manjaro), I do not have it setup as a DNS server or Web server, running from the command line and localhost, it works fine, but the path for the blog or post does not work for me, the first time I save it, it shows a sample line like: http://localhost:3000/2017/07/11, this will not work, so I have a URL, it points to my IP, but I can not use the external URL only localhost, normally not an issue, but having to hard code a URL, will not work, and when I save it, lets say I change it to, or something like it, I get: 2017/07/11/httpdomian-com2, making me edit it again, not sure what is up with that, I can file an issue, but it seems like others would have already done so, so my problem is that no matter what I change the path to, I cannot access it on localhost and cannot use the external URL, so its a stopping point for me, I cannot get past this problem. I did get the SETUP to work, I can get into the back end and everything seems to work fine.

    1. Update: I did get my server set up for development, I just got a new Router and Fiber 50 MB, I had to add my domain to host, and use setcap so I could use port 80, and it seems to work, I can access it via a Proxy, but had a friend verify that my issues where repeatable, and they were, so I added them to the issue section at github.
      I like the concept, now I can start to learn the Code and see how I can help, besides filling out issue reports on bugs, it will take me a while to get up to speed on the code before I can start making patches to fix the bugs I find.
      But your Really need a Forum, this is not really the place to fill this, nor is the issue section a place to discus issues, I know that wordpress has a forum plugin, I have it installed on one of my sites.

      Do you have a working Site that uses this CMS? If so, can you post the link.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s