Hi all!

I have a couple of months to create and deploy a small cluster for running docker containers.

The cluster will consist of 3 master nodes and some workers. When it is ready, it will consist of about 15 servers.

I have little experience with Docker (managing some containers on my home server), I have spent the last 4 or 5 weeks studying and testing with Kubernetes and I think it’s a little overkill for what it’s going to take. You run the risk of adding unnecessary complexity.

I am seeing that instead Docker Swarm seems easier to set up and manage.

To consider that I will be on my own to manage it.

What do you think?

Thanks!

  • rs5th@lemmy.scottlabs.io
    11·
    2 years ago

    I haven’t used Docker Swarm (I have barely used Docker Compose), but I have run a couple on-prem Kubernetes clusters (at my house and for clients at my day job) and cloud Kubernetes clusters, so I can speak to how complex it is it set up and run.

    My background is systems administration, engineering, IT, and now DevOps. I’ve been using Linux since Ubuntu 6.06.

    I set up my Kubernetes cluster with kubeadm because I wanted to learn, and it took me about a weekend to get my single master, two worker cluster up and running. I think you could probably do this using k3s much faster and have less learning curve (you don’t have to care as much about Container Network Interfaces, for example, because k3s makes that decision for you.)

    There is a lot of documentation out there on Kubernetes. Helm as a “package manager” (really a templating engine) can be nice if the software you want to deploy has a Helm chart that is well written. Writing your own Helm charts can be a learning process, I’ve modified some but not written one from scratch yet.

    Kubernetes releases new versions about quarterly. I’ve done several upgrades on my primary home cluster over the course of the past 2 years and they’ve been pretty smooth, about an hour of time investment total each. And remember, I’m on the more nerdy and complex flavor of Kubernetes. I think with k3s these would be even smoother and quicker.

    I feel like Kubernetes knowledge is probably more valuable out in the industry if that’s a factor for you. I haven’t come across any Docker Swarm clusters in my DevOps travels, just Kubernetes and some HashiCorp Nomad.

    I’m curious to see what folks say about Docker Swarm. If you have any questions about Kubernetes or running your workload on it, I’d be happy to try to help!

    • anthr76@lemmy.kutara.io
      6·
      2 years ago

      I agree with this. I think single node or not the industry is moving towards Kubernetes for container orchestration. Docker has showed their evil intentions and it’s time to leave them in the past. Even podman has native kubernetes manifest support (albeit limited last i checked) as @rs5th@lemmy.scottlabs.io pointed out there’s good avenues to take if you want to avoid the complexities of kubernetes like k3s.

        • anthr76@lemmy.kutara.io
          2·
          2 years ago

          I’m more of a Kubernetes the Hard way kind of person, but I think it can be suitable for certain production workloads. I’d trust a production workload on it way more then Docker Swarm

      • mlaga97@lemmy.mlaga97.space
        11·
        2 years ago

        I honestly hate that I can’t just deploy docker compose files directly to kubernetes seamlessly, and instead have to translate them to manifests. I’d say that having to drop all of that existing configuration as well as not being able to easily copy docker-compose’s for random new software I find is the biggest blocker for me being able to actually commit to using kubernetes.

    • bananess@lemmy.worldOP
      1·
      2 years ago

      Thanks for the reply!

      I also have a background as Linux Sysadmin but barely a couple of years.

      I have managed to get a Kubernets cluster up and running but I have difficult setting up containers. The Classic “whoami” works flawless by Traefik for example, but doing the same for Portainer seems like impossibile for me right now, going crazy :D

      I’m i choosing a difficult path? Is it possibile to have a good K8s cluster running nice without traefik?

      Thanks!

      • rs5th@lemmy.scottlabs.io
        3·
        2 years ago

        I use ingress-ngnix for all my ingress controllers, I’ve only messed with Traefik a bit in Kubernetes and it felt like it was fighting me the whole time.

    • nhgeek@beehaw.org
      1·
      2 years ago

      K8s all the way. That’s the general consensus in the industry. Swarm is not popular in comparison. My company sells a product in two flavors, SaaS and On Prem. Both run on K8s. We supply Helm charts and whatnot for On Prem installations (including in private cloud). There is good demand for K8s skill in the labor market.

  • johntash@eviltoast.org
    3·
    2 years ago

    I’m not really sure what the state of docker swarm is, but I’d recommend taking a look at Hashicorp Nomad too. It’s between swarm and k8s in complexity, but it’s been great to work with so far.

    Kunernetes is what I see most companies using, but it’s also usually overly complicated for what they need.

    What kind of workloads do you plan on running?

    • rs5th@lemmy.scottlabs.io
      51·
      2 years ago

      overly complicated for what they need.

      👀 glances at the 4 Kubernetes clusters running in my house / personal VPS

      • johntash@eviltoast.org
        2·
        2 years ago

        There’s nothing wrong with having an over complicated homelab! I manage dozens of kubernetes clusters at work and it really is a good skill to have for career reasons. My dig at it is mostly that a lot of smaller teams or companies will jump to it without really looking at their other options first.

        • rs5th@lemmy.scottlabs.io
          2·
          2 years ago

          Oh sure, I totally get it. Once I was fully bought in to k8s and GitOps for one environment, it just made sense to have my other ones use the same tooling.

  • mlaga97@lemmy.mlaga97.space
    2·
    2 years ago

    Docker swarm is pretty easy to set up and use (and lets you use compose files directly!) and is probably more than enough average self-hosters/homelabbers, but if you want to do something super fancy related to clustering there’s a good chance you’ll hit functionality walls quickly.

    Kubernetes is a pain to set up but is very flexible and ‘scalable’ to incredible levels, while being massive overkill for most applications.

  • Tupcakes@lemmy.world
    1·
    2 years ago

    docker swarm is def easier to manage, however I found it somewhat limiting. particularly around IP address assignment for services. I ended up using nomad, which is a nice inbetween. feels kinda like what swarm could have been.

  • Justin@lemmy.jlh.name
    1·
    1 year ago

    Kubernetes all the way. The Pod definitions look a little different from compose files, and the networking is a little strange at first, but it’s just better in every way.

    For one, you don’t get any of the dumb compromises that Docker Swarm has, and the industry has standardized on Kubernetes, so you’ll have much more options with Kubernetes.