Nothing matches that yet.
Nothing matches that yet.
It's no secret that one of the largest issues within Linux is program compatibility: too many variables, too many fragmented locations and too many package types. So why make it worse by creating another application package? The short answer: we didn't have much of a choice. The real answer is much longer.
When it comes to packages there are many acceptable answers available, however none of those answers fits DivisionOS and its goals: immutable applications, a shared resources and objects pool, and easy security patches, all shipped alongside a predefined behavioral system that can be expected to run on every copy of DivisionOS, and keep running for years without being repackaged.
Portable package formats ship with their dependencies baked in, which is generally great for portability and compatibility. However, this also means that no matter how severe the security vulnerability, it can never be patched, and the entire package must be deleted and replaced on every machine that uses it or has it installed. It's not just unlikely but near impossible to assume users will all self maintain their portable package updates. Some will argue the point: "portable packages are updated all the time." However, Johnny downloaded his package two years ago. It still runs great and he's never had an issue, but he also never reads CVE reports and isn't in a hurry to download an update. And what's worse, a vast number of portable packages are never distributed through a repo or store that would normally handle automated package updates, and there are an awful lot of users like Johnny. This creates a simple requirement for Division bundles: **we can't ship bundles with baked dependencies.**
Creating a new package format isn't exactly fun, and now that we have our first requirement, that dependencies must live outside the package itself, we run straight into our next issue, which is the direct reason portable formats exist in the first place: dependency collisions, better known in Linux as "dependency hell".
Imagine we have a handful of packages and they all use `libDFG`. Some may use `libDFG.so.1`, some may use `libDFG.so.2`, and some may have been built against two different copies of `libDFG.so.1` by different maintainers, neither of which actually matches upstream, with either program instantly crashing while loading the other's copy. Again, dependency hell. And what makes this worse is that we can't trust dependencies.
Some developers are great, and some, like anything else, are not so great. Imagine two developers, let's call them the LibXYZ team and the LibABC team. The LibABC team is amazing: they follow conventional versioning standards and proper upstream practices, and they never break dependency promises with malformed versions. And what's best? Let's imagine that because of this, their libs have been reliable and have never broken application compatibility in over 15 years...
Now let's consider the LibXYZ team. They just released `LibXYZ.so.1.5`. It's amazing and everyone loves it, and now we have ten thousand packages in hundreds of Linux repos that all require LibXYZ. After all, it's amazing, blazing fast and promises a lot. Unfortunately, six months after release, LibXYZ rewrote a handful of their functions. Many are just updates and bring major improvements, however FileArray was renamed FieldArray, and I-Bucket was removed because it was replaced by a better function. So what's the problem? All these changes build clean and have now been published as `LibXYZ.so.1.5`, the same `LibXYZ.so.1.5` that was released six months ago, and now over ten thousand packages in hundreds of Linux repos face imminent incompatibility.
And here's the real problem: from the outside, the `LibXYZ.so.1.5` from six months ago and the `LibXYZ.so.1.5` published today look exactly the same.. Both say 1.5, yet they are completely different.
::libxyz
The next issue is the FHS, or... the Filesystem Hierarchy Standard, the funny word being "standard". Let's take LibLFG. The LFG team properly places its plugins in `/usr/lib/liblfg/plugins`. However, some distributions move them to `/usr/lib/x86_64-linux-gnu/liblfg/plugins`, while others move them to `/usr/lib64/liblfg/plugins`, and the standard allows all of it. So what? Who cares?
::fhs
The short answer is your desktop cares, and the applications you run care. Now each version of an application must either be shipped in a portable package format or be built several times, once against the requirements of each distribution that runs it. This is why your favorite software is only packaged for some distributions and not yours. This is what's commonly and masterfully referred to as "Linux fragmentation". The FHS is a horrid offender, but not the single cause of it, and it's exactly why formats like AppImage, Flatpak and Snap exist.
Long story short, this isn't because developers suck and just "break things", it's because "some" developers suck and break things.
A more accurate statement is that systems are complex, and when things are flying fast, a bit of glass is going to get broken. Unfortunately, this glass often runs datacenters, major application infrastructure and yes, even your desktop.
This brings us to the biggest of our requirements: **never trust a library or dependency.**
Instead, bundles must use defined systems that cannot collide. This is the reason we've designed bundles in parallel with Division's Objects Store, with all bundles using state based objects rather than the trust based promise of `.so` versioning.
Rather than `LibXYZ.so.1.5`, Division uses `Name.ID.Collection.Hash`: four 20 character hashes that take a library or dependency and turn it into a state based object. Remember `LibXYZ.so.1.5`? The one that broke every package that was built against it six months ago? In Division that cannot happen, and here's why. Nothing in Division cares about a library's version number. What we care about is what the library actually offers, and how that's matched against its ID. Is `LibLMNOP.so.2.8` the same library as `LibLMNOP.so.2.8.1`? We answer that with a series of checks against what each one actually offers, its functions, not what its version number claims. If the answer is yes, the ID stays the same and 2.8.1 simply replaces 2.8's bytes. If the answer is no, a new ID is generated, making both copies of LibLMNOP in Division's Objects Store predictable, maintainable and fully patchable. In this example the answer was no, so we get two:
::objects
Notice both share the same name, the first field. To a program they are both just `LibLMNOP.so.2`. The ID is what tells them apart. Now we have two fully independent, byte verifiable and trustable objects that bundles can use at runtime, with zero collisions.
So what are these values? They equal `Name.ID.Collection.Hash`. Hash is pretty damn self explanatory, but I'll explain it anyway: we create a sha256 of the object so we can prove that it is byte for byte perfect and verifiable against our repo and build environment. The interesting one is collection...
Remember our issue with the Filesystem Hierarchy Standard? Short recap: people put things all over the damn place, and a program compiled by someone on one system may not exactly work on another. This is where collections come in. Quite simply, Division doesn't allow programs to drag dependencies from folders, because once again, we cannot trust them. Instead, we use collections.
The way collections work has less to do with the field itself and everything to do with our underlying system, "Prism". Without getting technical: when a program asks for `thisFolder/mylibs`, that request lands on the third field of an object's filename, "collection", and the program gets back that folder exactly as it was when the program was installed, one copy of every file in it and nothing else. Plainly, DivisionOS uses a layer of "Request and Response" rather than `readdir`. The folder a program reads was never sitting on the disk, Prism answers with exactly what that program was installed with.
And remember Johnny? On DivisionOS, the library his program uses isn't baked into his package, it's an object. When a security fix lands, that object's bytes are replaced once, same ID, and only after it passes the same checks against its functions. Every bundle using it is patched at the same time, Johnny's included. He never read the CVE report, he never downloaded an update, and he's patched anyway.
::johnny
So, the answer to the question "why did we make another package format?" is simple: because we had to.
DivisionOS has a series of goals that are very difficult to reach, and one of them is to chip away at fragmentation. Unfortunately, to do that we must first create further fragmentation. It's a bet on an idea, the idea that we can get to a world where you can install your favorite .deb, .rpm, .flatpak or .appimage all in one place, "albeit with a little disrespectful handling". A bet that software bundled today will still work on DivisionOS in five years without having to be repackaged. A gamble of lunacy, effort and a bit of engineering all combined.
That's why we created bundles, something only attempted by the clinically insane. And no, we're not done. We're still testing, still tweaking, and still crossing our fingers on this bet against our own insanity.
Archive