PackageKit aptcc backend refactored

Last week was a busy week, it’s funny how I mostly work under release pressure šŸ˜›

Well PackageKit 0.7.4 was to be released lastĀ Monday, so I decided to fix a few things I had planned, it all started to make aptcc emit the untrusted packages an installation is about to install, a thing I myself added to PackageKit but I’ve never managed to implement in my backend…

Currently if you use Apper or Gnome PackageKit, you choose a few packages to install/upgrade, and click “Apply” then what happens under the hood is:

  • First the simulateInstall() method is called to show you what is going to happen (packages to install, remove, update, downgrade)
  • You check the list, if you are ok you click continue.
  • Now installPackages(bool trusted = true) is called, This makes the PolicyKit auth prompts to install TRUSTED packages, and then PackageKit passes the packages to the backend
  • If the backend now detects any untrusted packages it fails with an error PACKAGE_UNTRUSTED_REQUIRED
  • The GUI has now to warn the user about untrusted packages being installed, but it don’t give any clue of which packages are actually untrusted…
  • The user accepts the untrusted dialog and a new PolicyKit auth prompts him to install UNTRUSTED packages. (note that this is the second dialog).
  • last installPackages is called on the backend with bool trusted = false, and the backend ignores the trusted check.

As you can see we have two issues, first two auth prompts, second you can’t know which package is causing the untrusted issue.

Some time ago I half fixed the issue by adding a new enum to enable backends to emit untrusted packages on simulate, this way the GUI could know before hand which packages are untrusted and that installPackages() must be called with UNTRUSTED = false. This way we show the user which packages are untrusted and prevent the first useless dialog.

With the new PackageKit release I fixed the other hald by implementing this support on aptcc, but as you guessed from the post title I didn’t stopped there šŸ˜›

It turns out that I wrote aptcc back in 2008~2009 at that time I didn’t know anything about apt-pkg API, and of course didn’t had the experience with C++ I have now (not that is that much), so looking at the code I started to see a bunch of things that could be improved, and by improving I started to break a few things but fixing would end up rewriting a bunch of parts, the final result is probably a code much cleaner probably faster (since I found out I didn’t had to store a pair<pkgname, pkgversion>) but most importantly I fixed several potential bugs, and apt pinning using pin.d dirs now works exactly like apt-get.

Almost forgot, aptcc now has Support Filter (thanks to Matthias Klumpp), SimulateRepairSystem() and RepairSystem() support, which means no more apt-get -f install šŸ˜€ Ā , making it back again the fastest and most complete apt backend.

Take a look at the huge commit list:

– aptcc: Add a new class to show cache opening progress (Daniel Nicoletti)
– aptcc: Add AptCacheFile to handle all pkg cache openings (Daniel Nicoletti)
– aptcc: Add SimulateRepairSystem and RepairSystem methods (Daniel Nicoletti)
– aptcc: Avoid crashing when error messages containing invalid utf8Ā is sent (Daniel Nicoletti)
– aptcc: Change the way we mark auto-installed packages (Daniel Nicoletti)
– aptcc: Correctly emit if a repo is enabled (Matthias Klumpp)
– aptcc: Create a PkgList class to provide convience methods (Daniel Nicoletti)
– aptcc: Don’t emit error if repo is empty (Matthias Klumpp)
– aptcc: Don’t emit untrusted packages when we are not simulating (Daniel Nicoletti)
– aptcc: Emit newly-installed local package information (Matthias Klumpp)
– aptcc: Emit packages that are untrusted with theĀ PK_INFO_ENUM_UNTRUSTED enum (Daniel Nicoletti)
– aptcc: Emit UnfinishedTransaction when we were not able to fix theĀ cache (Daniel Nicoletti)
– aptcc: Fix a multiarch bug that failed to resolve packages (Daniel Nicoletti)
– aptcc: Fix crash when a package for an invalid version (Daniel Nicoletti)
– aptcc: Fix INSTALLED filter (Daniel Nicoletti)
– aptcc: Fix resolving of gdebi packages when they have :arch fieldĀ (Daniel Nicoletti)
– aptcc: Fix SUPPORTED check and component-finding code (Matthias Klumpp)
– aptcc: Implement basic support for InstallFiles() (Matthias Klumpp)
– aptcc: Implement SUPPORTED support (Matthias Klumpp)
– aptcc: Improve classes readability, and fix a crash caused by anĀ invalid package (Daniel Nicoletti)
– aptcc: Improve the autoremove code (Daniel Nicoletti)
– aptcc: Make sure BuildPolicy is called when getting the dep cacheĀ (Daniel Nicoletti)
– aptcc: Make sure we discard errors when closing the cache (Daniel Nicoletti)
– aptcc: Mark dependencies of a file-installation as automaticĀ (Matthias Klumpp)
– aptcc: Move cache related functions to pkgCacheFile subclassĀ (Daniel Nicoletti)
– aptcc: Move ShowBroken to AptCacheFile, and added CheckDeps fromĀ apt-get (Daniel Nicoletti)
– aptcc: Refactored to use AptCacheFile (Daniel Nicoletti)
– aptcc: Reident the code and re-enforce KDElibs coding style (DanielĀ Nicoletti)
– aptcc: Remove a code duplication and emit proper RestartRequiredĀ signal (Daniel Nicoletti)
– aptcc: Reorganise parts of AptCC to make the code more readableĀ (Matthias Klumpp)
– aptcc: Separate the try install/remove functions (Daniel Nicoletti)
– aptcc: Simplify code for checking trusted packages (Matthias Klumpp)
– aptcc: Some more stuff required for installFiles() (Matthias Klumpp)
– aptcc: Update license headers & fix some more indentation and typosĀ (Matthias Klumpp)
– aptcc: use const_iterators to avoid mistakes (Daniel Nicoletti)
– aptcc: Use _exit() to quit the child process as synaptic doesĀ (Daniel Nicoletti)
– aptcc: We are not apt-get so remove “APT::Get::*” confs (Daniel Nicoletti)
– aptcc: When on multiarch some packages that should have the :arch
– Add full integration so that UI can know when to repair the systemĀ (Daniel Nicoletti)

Stay tunned for the next Apper release šŸ˜€

PackageKit aptcc backend refactored

print-manager is now low latency and low CPU demanding

First of all I really want to say a big: Thank You!

For all of you who donated to my fund raisingĀ campaign, the goal was reached pretty fast and there was also people sending the money without PayPal, if you are still willing to do the contribution there’s still 5 days left, and if you can’t make a donation I already appreciate the good feedback I receive, you CAN still send patches fill bugs, translate there are lots of ways of contributing.

Back to the topic…

It all started last week when I was trying to hunt down a ghost, yes a ghost!! You don’t believe me? Take a look:

Ghost Job

If you runned our new plasmoid for a while, I don’t know how much time is needed, if suspend/resume has any interference in it, but it turns out a ghost job and printer will show up… Debugging this is really hard because when I put the plasmoid the ghost is not there, so I don’t know how to reproduce it quickly, but I know the cause.

Let me first tell how all print-manager worked before this change

CUPS is like an HTTP server, you create IPP requests, send, and get back a response. So when the application/plasma data-engine started I asked the server for all printers, then I asked for all jobs. This process is a bit CPU consuming because it has to convert Qt types (QString, QStringList..) to C types but also adding then with the proper CUPS flags (ie say a boolean var is bool), and do the opposite when we get a response.

Now when the UI needs to refresh I actually had to do all of that using a timer every 1,5 second, plus update the model/plasma data-engine, and even check if anything changed to avoid refreshing the UI which is increases the CPU usage.

Because of the ghost problem I remembered seeing some DBus interface on printer-applet, so taking a look at the code I found “com.redhat.PrinterSpooler” Interface. Using dbus-monitor I was able to create a new printers data-engine to test how it would work, and turned out that this was a very good change, now instead of refreshing all printers using a timer, I could not only refresh the list when something changed but also only ask CUPS the printer I was interested in.

So say if you click on pause printer immediately a signal is emitted and the UI will refresh, thus creating a very low latency. Now not happy enough I had to do the same with the print jobs, but I found out thatĀ “com.redhat.PrinterSpooler” didn’t provide enough signals. Say I cancel a job using http://localhost:631 no signal is emitted resulting on the print-queue thinking the job is still active, you can find this bug on printer-applet.

I decided to try the Gnome Control Center printers module, and yes the bug was there too, so I decided to contact the RedHat guy (Tim Waugh) to see if we could get another CUPS patch, and it turns out thatĀ “com.redhat.PrinterSpooler” was not used long ago, and there was a new “org.cups.cupsd.Notifier” interface, which for my surprise Gnome printers Ā module used.

The new interface is quite different from the old one, instead of just waiting for a signal to arrive, we have to actually register which signals we want to listen, and CUPS returns a subscription ID, this ID has to be renewed from time to time (but surely is much less work than getting all your jobs and printers from time to time).

Reading the CUPS patch that makes this working I found out why the Gnome thing wasn’t working, it didn’t registered to the needed events, so now not only KDE print stuff works better but also Gnome’s šŸ˜›

If you read all of the above you are still curious about the ghost, aren’t you?

Well as I said I don’t know when it happens but I sort of know why, the old data-engine had “Printers”, “ActiveJobs”, “AllJobs” entries which didn’t had any valid data, but where used to probe data-engine to update it’s entries. So for some reason these invalid sources showed up on the listing, I even tried to use a DataModel filter but it didn’t resolve.

With the usage of DBus I dropped two features, listing all jobs or completed jobs. The problem is that these lists tend to be too big, and plasmoids start to get slow to fill the data, also to get rid of the ghost entry I removed the invalid entries so there’s no way of telling the data-engine what do you want. The alternative would be to have more two data-engines, but 99,9% of the users will care only about the active jobs which this will fit the need perfectly (the QWidget GUI is also available providing this feature).

To end this post, I have a nice screenshot:

print-manager plasmoid using swicthes

As you can see I’ve changed the plasmoid to use switches to pause/resume printer.

Make sure you make uninstall before testing the new version since the old data-engine was removed.

Enjoy!

 

UPDATE: The above screenshot with a plasmoid smaller and printers grayed when disabled:

print-manager is now low latency and low CPU demanding

colord-kde 0.2 released!

Today is the 0.2 release day!

I’m very excited about the overall feedback that I received from the 0.1 release, a fix here, a bug report there, and we finally have a real solid release. System Settings bug was fixed, monitor plug’n’play was improved, the profiles selection bellow the device item doesn’tĀ flicker anymore, most (if not all) Krazy issues where fixed and a few new features where added.

The coolest new feature is calibrate button, as I explained on the first post we won’t have a KDE interface so soon as the code has to be madeĀ share able, so it’s not the best solution, but the button I added will be enabled only if gcm-calibrate is available:

Calibrate button

As you can see beside the obvious GTK style change, is that it actually integrates pretty well as the dialog sets System Settings as it’s parent:

GCM Calibrate

Thanks to Richard Hughes who sent me a freeĀ colorhugĀ device, I was able to play with it and make the button sensitive when you connect the device, so if you don’t plug it on the USB port the calibrate button will stay disabled. Besides the colorhug colorimeter which is meant for LCD/LED devices, the gcm-calibrate can also calibrate others devices without a measuring device like printers.

colorhug device calibrating the screen

Enjoy!

UPDATE: Thanks to Tom Albers the downloads are now on KDE repos šŸ˜€

Download: Ā http://download.kde.org/stable/colord-kde/0.2.0/src/colord-kde-0.2.0.tar.bz2

A big thank you for all that already donated to my found raising campaignĀ it almost reach the goal, in just 2 days!

colord-kde 0.2 released!

libdebconf-kde 0.2 released!

This release is mostly to help packagers, I delayed it a lot because of theĀ bugĀ 255935, it turned out that as kpackagekit-smarticon and apper-sentinel crashed when exiting showing the debconf stack I thought the problem was actually in libdebconf, so today I decided to dig some old bugs and fixed this crash in Apper (hence the bug).

This new release include some nice changes made byĀ Modestas Vainius which among other things adds support for FIFO pipes on the debconf-helper command line and implement X_PING support in it.

So I added a few Krazy fixes and we are done as the bug was not on libdebconf-kde.

Enjoy!

And if you didn’t read my last post on theĀ fundraisingĀ campaign (thanks a lot for those who already contributed), please take a look it’s almost there šŸ˜€

http://www.indiegogo.com/project/widget/87790?a=525675

libdebconf-kde 0.2 released!

Help Apper, colord-kde and print-manager by donating

Short Summary

As some of you know I develop mostly KDE stuff, but last year I had aĀ loss.Ā So when I moved to Argentina I sold a bunch of things and had to buy them again when I came back to Brazil, there are two still that I couldn’t afford: A printer and a monitor.

Stuff I develop: Apper, colord-kde, print-manager, libdebconf-kde, packagekit-qt2 and aptcc

What I Need & What You Get

The monitor is really important so I can use my desktop PC wich is far more powerfull than my work laptop, using my desktop allows me to run several VMs to test the different PackageKit backends and make sure Apper runs fine.

The printer is also important because as you know I also develop print-manager which got some real cool improvements but still misses some stuff, though it’s not completly required to have a printer and do this development (which most of the time was done without real printers), I’m now in the adding a new printer phase, so yes I need a printer.

In short:

The monitor helps me mostly with testing my stuff on different distros and different setups.

In order to complete print support

The Impact

To understand the values:

Monitor Dell 23″Ā costs R$ 799,00 -> ~400,00 US$ (the same one is 250$ in US, Brazil and it’s abusive taxes..)

Samsung SCX-3005w costs ~ R$ 450,00 -> ~225,00 US$

I’m completely happy if I just reach the goal, so I can be more productive in my codings, if the goal is reached I’ll consider your donnation as a personal help (which of course will help me keep coding in KDE).

Other Ways You Can Help

If you can’t help by donating that’s ok, I still need people to test, code and even write docs.

print-manager and colord-kde need especial testing since it doesn’t have much users.

Thanks!

http://www.indiegogo.com/project/widget/87790?a=525675

Help Apper, colord-kde and print-manager by donating