My Opera is closing 3rd of March

Ruarí's thoughts

Package and dependency management shouldn't put you off Slackware

, , , , ,

Since I have been running Slackware as my primary distro I've heard the occasional comment along the lines of, "Manual dependency management must be a real hassle" or worse, "How do you work without a package manager?". Rather than repeat my response ad nauseam individually to those asking such questions, I thought I would save myself the trouble and type up my thoughts once publicly, so that I could simply point them to this post. wink

Package management

Let's start with the second suggestion first, because it is totally false. Slackware does have a full set of tools for managing packages. The only obvious 'missing feature' is dependency management, which some people incorrectly equate with package management. I'll explain why this one seemingly important missing feature isn't as big a problem as you might think a little later. First, lets remember that package management is actually a much broader subject and also encompasses many other important tasks such as download, installation, logging/tracking, upgrading, removing and package creation. The package management tools provided by Slackware perform all of these other tasks with aplomb and even provide several advantages over some of the more popular Linux package managers.

What are these advantages? Well firstly, all logs are written in plain text, in a way that is easily read by humans but still machine-parseable. This means I am not just limited to the Slackware provided tools. I can run queries against them with a wide variety of standard UNIX utilities directly. I can read logs with less, cat or any text editor, and I can process, manipulate and collate packaging data with grep, awk, python, perl, etc. Or to put it another way, I am not forced to use any distro specific tool to gather (or even change), the information stored about my package setup.

Similarly the packages themselves are equally open, since they are simply compressed tarballs containing the filesystem contents and some optional text files that describe the package and perform any needed post-installation steps. You can view, open and even edit packages with a wide selection of tools (GNU tar, BSD tar, perl, etc.). Indeed if you really wanted to you could even create them without using Slackware's provided makepkg tool¹. Though I must strongly stress that last point really isn't recommended, as admittedly there are a couple of potential 'gotchas' related the exact listing of the internal file path and dealing with symlinks. Still, it is possible! For comparison consider the RPM format, which effectively requires installing rpm and associated tools for any reasonable level of query or manipulation. Installing rpm on non-rpm based systems can range from being a minor inconvenience to being an almighty hassle, or even down right impossible, the further you move beyond the world of Linux-based OSes. Even deb, which is vastly better in this regard, still uses the relatively obscure (outside of Unix-like systems) ar format as its outer container.

Finally, perhaps the most interesting difference is that all of the packaging tools are implemented as simple shell scripts. This greatly lowers the barrier of entry to tweaking them, bringing it within the bounds of a user with intermediate to advanced Linux knowledge, rather than a Linux 'Guru'. For example, prior to the change of default Slackware package compression format (from gzip to xz) it was not uncommon for some Slackers to hack in support for alternative compression formats themselves. How many deb or rpm users (not developers) do you know who have hacked and adjusted their package managers to better suit their own personal needs or tastes?

I think the point to take away from all this is that one of the upsides of using a system like Linux, is that it is supposed to empower users and give them greater control of their system. In this regard Slackware's package management tools excel.

Why dependency management doesn't have to be difficult

So ... dependencies! bigsmile This is probably the best known difference between Slackware and other popular distributions. Slackware doesn't even attempt to automatically handle dependencies during package install, leaving it up to the user to figure out what combination of other packages must be installed. From the outside and recalling some of my own preconceptions prior to moving to Slackware, I can see how this would appear to be a major omission.

One of the main reasons that Slackware doesn't bother with automated dependency management is that it doesn't need to. The person installing can do that and it doesn't even have to be that hard. Whilst it is possible to completely customise the selection of installed packages, users who are new to Slackware are recommended (by the accompanying documentation) to start with a full install (i.e. every package). And in fact, there are many seasoned Slackware users who also do this, even if they are fully capable of pairing down the package selection to something more minimal. Obviously a full install means that dependencies are completely satisfied. See, I told you Slackware was simple! p

I realise a few people might now be thinking, "Well isn't that just too simple, treating the distro like a monolithic system, rather than the modular entity it is?". I agree, indeed it makes you wonder why every other distro had to make things so hard. wink

Now of course some people reading this are probably already thinking up a few potential problems with this approach. I can make a fair guess at what these potential problems are because I have heard them before. There two most common arguments I have seen for not wanting to do a full install are, "it wastes space" and "it bloats the system, slowing it down". The reason for assuming these will be issues, is often based on experiences with other distros. Once you understand a little bit more about Slackware's way of doing things it soon becomes obvious that neither are likely to have much impact on you if you choose to use Slackware.

Lets start with wasting space. According to the back cover of the official install DVD of version 13.37 (the current stable release as I write this), a full install of Slackware occupies 6.8Gb. Whilst this might seem like a lot to those of us who have been using computers for a number of years, in modern terms it is almost nothing. Consider the fact that even amazon.com (not the cheapest place for electronics) sells decent 500Gb disk drives for less than $50 (USD), meaning that 6.8Gb of space costs less than $0.70! Now I will admit that a hard disk is one of the cheapest ways of adding storage and isn't always feasible on smaller laptops or netbooks. However on the other hand, the cost is still very low even when choosing more expensive ways of adding storage to your system. For example, Amazon also offers new 8Gb microSD cards for less than $5 (USD). So you have to consider is it really worth your time to even consider cutting down the install selection, at least from a spacing saving perspective?

This obviously begs the question, "so wouldn't you always do a full install for all distros?". There are two parts to this. Firstly some distros (such as Debian or openSUSE) have very large repositories, packing just about every open source application available (including a lot of stuff that only a handful of people will never even consider using). So if you did this you really would start to waste a lot of disk space. Perhaps more importantly the system really would start to slow down due to all the bloat.

On Slackware a full install running a lightweight window manager like Fluxbox will run every bit as fast a slimmed down install running Fluxbox. Why? Because of its simple design philosophy. Slackware does not make any assumptions about what the user intends to do with his or her software. This is unlike the majority of other distros, which do make assumptions and try to automate as much as possible in line with those assumptions.

This means that on most distros if you install some application that runs as a daemon/service, the install scripts in the post install of that package will typically start it immediately after installation finishes. This would appear to make good sense as why would you be installing it otherwise? Whilst it is indeed most likely that you would want it started, there are other possibilities and for this reason Slackware doesn't attempt to second guess the answers to questions like these, leaving it up to you. For example, you might not want it running all the time or you might have installed the package only to access one of the other binaries or libraries installed alongside the main program. Who knows, you might even have installed it because you couldn't find documentation on the author's website and wanted to read the included files or man page to find out more. The result of all this is that unless you enabled a service during the Slackware install, or later manually, most of the software installed just sits on your disk doing nothing other than taking up space and hence has no affect on the speed of your running system at all.

So yes, a large amount of software may be left just sitting around doing nothing, but you know what? It has little (if any) real negative affect on me, so why would I care? Indeed you never know when one of these applications might turn out to be useful. I don't typically want to run a full featured web-server on my desktop but should I ever need to test something that might require it, I have Apache installed and ready to fire up! Indeed this simple way of not having to think about dependencies can sometimes be a real advantage. If I want to try some program or utility at some point in the future I quite often find it is already there, negating the need to search via the package manager and then install it. This is particularly nice when installing an application would have pulled down hundreds of Mbs of extra dependencies. I might not care about some used disk space but I do care about wasting my networking connection if I am in the middle of using it or I am on a slow network.

And what about dependencies for non-official packages? Well most of the popular third party/non-official Slackware repositories provide some means to simplify dependency resolution, sometimes this is done by providing a tool to deal with the inter-dependencies found within that repository, or other times by simply listing any non-official (by that I mean not included in the standard repository) dependencies in README files provided with each package. This is the approach taken by what is probably the most popular third party repository these days, SlackBuilds.org. It works because Slackware installs such a good selection of common libraries, which means that many of the applications already have the majority (if not all) of their dependencies fulfilled by a complete Slackware install. Since very few extra packages are needed it doesn't tend to be the hassle many might expect. And for the really lazy, using helper tools such as sbopkg (and its queuefiles system) you can even fully automate the process.

So there you go, not only does Slackware have a decent set of package management tools, the one area that tends to scare people off (dependency management) doesn't have to be daunting at all! bigsmile

¹ If you really did want to create your own Slackware packages without makepkg (for whatever reason), here is a short summary of how it could be done. Switch to the root of your unpacked package contents and check for symlinks: "find . -type l". If they exist convert them into shell script code appended to doinst.sh: "find * -type l -printf '( cd %h ; rm -rf %f )\n( cd %h ; ln -sf %l %f )\n' -delete >> install/doinst.sh". Finally, create a compressed tar archive with version 1.13 style directory formatting, either by using GNU tar-1.13 directly (it is installed on Slackware) or by forcing modern GNU tar to emulate this style of formatting by using a transform: "tar caf /tmp/some-application-1.0-x86_64-1.tgz . --xform='s,^\./\(.\),\1,' "

Watching YouTube without Flash via MinitubeHow to use multiple Opera profiles with just one installation, using the -pd (personal directory) switch

Comments

Audrius Kažukauskasaudriusk Tuesday, September 27, 2011 10:11:23 AM

Another thing that people using some other distros often don't know is that because Slackware doesn't manage dependencies, there's no need to split packages into -devel, -doc, -utils, etc. You get everything (development headers, documentation, ...) inside a single package (as often intended by authors of particular software). This greatly reduces the number of packages and further simplifies things.

Ruarí Ødegaardruario Tuesday, September 27, 2011 10:44:29 AM

Indeed, this means that whilst the official Slackware repository is smaller than many other distros, it is usually easier on Slackware to use a source package directly from the author, i.e. "./configure && make && make install" is far more likely to work right out of the box.

Handy for testing those interesting little applications one stumbles across from time to time, that don't seem to be in any repositories.

Ruarí Ødegaardruario Tuesday, September 27, 2011 10:54:11 AM

That said, hassles with source packages tend to be bigger on the so called 'newbie friendly' distros like Ubuntu and derivatives. And this I suppose is fair because the maintainers of those distros don't expect regular users to frequently compile from source. The other distros where the users are expected to regularly compile from source (e.g. Gentoo, Arch, etc.), don't tend to have such problems either. wink

Ruarí Ødegaardruario Tuesday, September 27, 2011 11:25:23 AM

I should probably also add a little extra comment regarding doing a 'full install'. Whilst this is recommended and generally has few real world downsides, I am not trying to imply that: it is the only option, that every Slacker does this, or that there is never a good reason to do a slimmer install. The installer allows you to completely customise your install, many Slackers make use of it and there are certainly valid reasons for doing this. An example being if you were installing on exotic hardware where disk space cannot easily be added (e.g. some of the small ARM-based systems) but this is far from the only reason. Others such as 'to see what is possible', are equally valid.

All I'm really trying to get across is that in the majority of causes a 'full install' is a viable option on Slackware and shouldn't be immediately discarded as a possibility, as you might do with other distros. I have often seen comments on forums from users who are new to Slackware, where they try and start with something minimal because they think they should, rather than because they have a genuine need. It is a shame if they then encounter problems because it really doesn't need to be that hard!

Even if you do plan on performing a striped down installation, if you are new to Slackware, I still think a 'full install' is a good place to start. It allows you to play around with removing packages and seeing what happens. This gives you a chance to build up knowledge of how things work together and is a lot more sensible than attempting to do a nice slim 300Mb install at your very first attempt. Also, one good thing about building up knowledge slowly rather than doing a rush job, is that what you learn is more likely to stick.

Finally, it is worth remembering that whilst a slim install is undeniably harder than many other distros the very first time you attempt it, each subsequent small install will get easier and faster as you learn the interrelationships between packages. Once you have gained this knowledge, a small install can be done relatively quickly, especially if automated with tag files.

Ruarí Ødegaardruario Tuesday, September 27, 2011 12:21:24 PM

It may sound obvious but my personal tip for those looking to slim down the install would be to start by considering the big packages that you understand, rather than trying to take out a few tiny utilities and drivers (that may only be a few Kb each) or by removing libraries that you have never heard of (but could turn out to be crucial).

For example, you could consider dropping the whole KDE series if you aren't a KDE fan to save around 900Mb. Another good option might be the 415Mb (installed size) kernel-source package. Whilst you will need this package if you want to compile a custom kernel or if you need to compile special kernel modules (like the VitualBox Guest additions), it can be a nice saving if you don't need it. Even if you do need it you can still consider removing after you are finished with it, as it is the type of package you typically only want at the beginning, when first setting up your system.

You can easily take the install size down to less than half by just removing a few of those big well known packages. And remember it is the packages' install size you care about. kernel-source is only 62Mb compressed, so you might not have considered it if you were only looking at the size of the compressed package.

I'd also personally favour removing as little as you can get away with, so that you can continue to enjoy the benefits of having a nice varied collection of solid packages to hand should you ever need them.

For other view points and tips, a search for 'minimal slackware' in your search engine of choice is likely to lead you to some useful resources.

slackwrdave Tuesday, September 27, 2011 10:05:10 PM

I thoroughly enjoyed this post and comments. Nice job.

Manpreet Singh DhillonManiDhillon Tuesday, September 27, 2011 11:32:08 PM

Nice article. You thoroughly discussed what most of the Linux users are concerned about in Slackware, dependencies.
Though in my humble opinion managing and making a minimal working install of Arch is quite fast.
Though Arch and Slackware work on the same philosophy that of being user centric not user friendly. I am not an expert on Linux but I am using different Linux distros since 2006 but still I sometimes got many problems in Slack and thus changed to Arch for same needs.

Corey Mwambacoreymwamba Wednesday, September 28, 2011 3:42:43 AM

Thank you very much for the pointer to this Ruari!

I actually think I'm in a position to switch now - It's not even as if I'm using the openSUSE kernel any more... and if Slackware just uses tarballs then it removes a layer of complexity.

Time for a back-up, reset the SSD, then install, wish me luck! smile

Mad Scientist (عادل)qlue Wednesday, September 28, 2011 4:42:26 AM

Now I need a second machine for experimenting on! sherlock.

Ruarí Ødegaardruario Wednesday, September 28, 2011 7:15:13 AM

@coreymwamba: Good luck! bigsmile

Also, here are a few tips for you, if you don't mind. wink

The simple text file documentation provided with the install media is actually really helpful on Slackware and is regularly updated. I would encourage you to actually read README.TXT, Slackware-HOWTO, CHANGES_AND_HINTS.TXT and README.initrd. These files aren't that long and should prevent you from making typical mistakes if you follow them closely.

In addition here are a couple of links you might find useful when you first set off on your Slackware adventure:
http://wiki.linuxquestions.org/wiki/Slackware-FAQ
http://duganchen.ca/writings/slackware/setup

Also just a few quick tips of my own regarding things that IMHO seem to be brushed over a little too quickly in some of the other documentation and/or guides that I have read.

If you want to know anything about the packages on your system look at the package logs, which tell you version numbers, install and package sizes, package contents, etc. you can read them with less, a text editor or check specific things with grep. The main ones are found in /var/log/packages/. The post install scripts (that were run after a package was installed) are in /var/log/scripts/. Also removed packages have logs in /var/log/removed_packages/ and /var/log/removed_scripts/

Remember that Slackware uses a BSD-style init system. All of Slackware's native startup scripts are in /etc/rc.d/. Typically scripts that are executable are the ones that are run on startup, which means that if you remove the executable bit (i.e. 'chmod -x rc.scriptname'), they won't run on startup. Or add it ('chmod +x rc.scriptname') if you do want them to run. Easy, eh? If you need to start some extra system wide service that you have installed from a third party repository, and it either doesn't provide a startup script in this location or does but is not automatically run once its script is made executable, then add an entry to rc.local to call it (and perhaps one to rc.local_shutdown to have it cleanly stop when you shutdown). That is about it for starting and stoping services on boot.

Also, since Slackware provides very few system wide configuration tools, you are going to want to get familiar with the files in /etc because everything is configured in these files when making system wide changes.

If you are unsure about anything and can't find documentation check out Slackware's official forums, which are actually hosted on Linuxquestions.org. They are a good bunch of knowledgeable people (Patrick Volkerding himself occasionally even answers people) but they do expect you to do your homework so run a search first before you ask! wink
http://www.linuxquestions.org/questions/slackware-14/

Finally, the following Blog is quite interesting to keep up to date with what is going in the Slackware world:
http://slackblogs.blogspot.com/

@qlue: You can always test under VirtualBox. wink

Ruarí Ødegaardruario Wednesday, September 28, 2011 8:09:13 AM

@coreymwamba: How could I forget. Most important of all you will want op2slk to convert Opera Next tar packages into Slackware format. wink

Or if you prefer I put up pre-converted copies here.

Mad Scientist (عادل)qlue Wednesday, September 28, 2011 12:33:31 PM

@Ruari I only have a netbook. It's just not capable of virtualisation. p.
(the Intel Atom is just a glorified 386 masquerading as a 686 after all. wink

Cutting Spoonhellspork Thursday, September 29, 2011 1:25:08 AM

Originally posted by qlue:

the Intel Atom is just a glorified 386 masquerading as a 686 after all.


More like a P2 with faster gates/more cache, several forms of SSE and Hyperthreading. There's a special "Atom" config set on several newer development kits, especially regarding multimedia. Yes it's still "just" an Atom, but 4 virtual cores burning up at 2.53GHz don't feel very slow.cool

Ruari: If I've set Opera to "Open" .bz2, .gz, .tar and .xz files by automatically plugging them into command switches for untar/install.......does that make me......evil?

Mad Scientist (عادل)qlue Thursday, September 29, 2011 2:29:11 AM

That's the beauty of choice. p.
Some hard-core Debian users would cringe at my choice of using the non-free Opera on my Debian based Crunchbang operating system. rolleyes.

Cutting Spoonhellspork Thursday, September 29, 2011 2:35:31 AM

And I shudder at the thought of slowing down Crunchbang with five+ other programs to replace everything Opera does.scared

Mad Scientist (عادل)qlue Thursday, September 29, 2011 2:48:18 AM

Actually, Opera is far better suited to creating a cloud OS than Google's Chrome browser. rolleyes. Opera has built-in bit torrent support. A very good built-in email client (which I only use for rss feeds) and Opera link. Then there's Opera Unite. You could do about 60% of all basic office computing with Opera and a good network connection. party.
Can you imagine what would emerge if Opera Software were to release their source code under a Gnu licence? bigeyes.

Cutting Spoonhellspork Thursday, September 29, 2011 4:16:52 AM

A big gap will be closed if/when Opera supports filewriter API and henceforth completes the majority of FileIO/Drag-n-Drop support. Really....it does feel that the desktop product is tantalizingly close to a one-stop solution. Google claims to have already achieved this but ChromeOS is embarrassingly deficient in several key areas when denied network access. Meanwhile Dragonfly demonstrates how amazing a hybrid online/offline application could really be.wizard

And now Opera has the "TV Store" or whatever for their new SDK. This implies hybrid applications, more powerful widgets or extensions, possibly some form of DRM support. With a few added features from the Desktop product, Opera TV could be equal or superior to ChromeOS in many different ways.

Ruarí Ødegaardruario Thursday, September 29, 2011 6:41:16 AM

He he ... guys, I am pretty lax on my blog and really don't mind if people go a little of topic but we have strayed quite far from packaging and dependency management. p

Ok, to bring it back on topic, here is a trick that a user with a full Slackware install can use to see what his or her largest (install-size) packages are. It'll help get some ideas about what to consider removing if you do need to conserve a little space.

Issue the following to produce a sorted list of all packages at least 1Mb in size:
$ grep "^U.*M$" /var/log/packages/* | sed -r "s|.+/(.+):.+: +(.+)|\2\t\1|" | sort -n 

If you are not a regex fan, here is a different way that is a bit easier to read:
$ (cd /var/log/packages/ ; grep -x "U.*M" * | awk -F: '{print $3 "\t" $1}' | sort -n) 

Ruarí Ødegaardruario Thursday, September 29, 2011 8:17:37 AM

Originally posted by ruario:

For example, prior to the change of default Slackware package compression format (from gzip to xz) it was not uncommon for some Slackers to hack in support for alternative compression formats themselves.

Just an extra point on this. I still do this myself, not to add stronger compression but instead faster compression. My own installpkg and upgradepkg are hacked to support the extremely quick lzo compression method because I often convert (using an adapted op2slk) various internal builds of Opera Next to this tweaked Slackware format for my own use. I'm more concerned with the speed of conversion, installation and upgrade than the size of the packages themselves. In fact, I usually delete the actual packages immediately after installation as I know I can just recreate them later if needed.

Ian Murrayianmurray Friday, September 30, 2011 2:12:02 AM

It has never put me off.In fact I love it for that.If ./configure finds a dependency that really need to be there OK I will get it.

Unregistered user Tuesday, January 14, 2014 2:51:16 AM

Geckgo writes: Thanks for this article. I've had slack running in a VM for a couple weeks now. I'm making an automated install so that when I switch to slack full time on my laptop, I can just run a couple script files and be done with the whole thing, completely packaged and configured the way I want, right down to how new user /home spaces are set up, like my own micro-distro. And if I thrash my system I can have another identical one up and running less than an hour. (sometimes I get lazy when compiling installing things like games and miss something) Just wanted to say, that while far from minimal, /a /d /l /n and /x will give you a nice base system to work with. The only other "dependencies" I've needed so far are select picked from /ap /xap and ../extras and they are part of my scripts :) Doing it this way is teaching me a ton about slackware and linux in general. Yesterday was the first time that I realized colorful ls commands are a result of a bash alias and not the terminal editor :P Learning something new everyday :)

How to use Quote function:

  1. Select some text
  2. Click on the Quote link

Write a comment

Comment
(BBcode and HTML is turned off for anonymous user comments.)

If you can't read the words, press the small reload icon.


Smilies