DevOps ist als Buzzword gerade in aller Munde – jede IT-Abteilung, die etwas von sich hält, und jeder Manager oder Engineer, der „Project Phoenix“ gelesen hat, will es umsetzen, denn die Probleme sind allbekannt und die Lösungen doch offensichtlich einfach.
Vor einem Jahr gründeten Michael Dombek und ich, Karsten Samaschke, die Cloudibility. Wir hatten einige Ideen und Erwartungen in Bezug darauf, wie sich unsere Firma entwickeln soll und was wir unseren Kunden anbieten wollten.
Die schlichte Wahrheit jedoch ist: Wir lagen sowas von falsch damit!
A year ago, Cloudibility was founded by Michael Dombek and me, Karsten Samaschke. We had some expectations in regards to how we wanted to grow the company and what we wanted to offer to our customers. Truth is: We were plain wrong!
This is, what you have to take care of when trying to establish a DevOps-culture: Many, if not most, people being affected by a DevOps-approach, will agree with it. At least as long as it does not force them to change anything and as long as they can sell it to their customers or supervisors since DevOps is very popular these days and is to be implemented by them, not by themselves.
Just last week, we had such a situation: We are in a short-term project with an enterprise customer. We have been hired by the vendor doing operations for that customer in a somewhat-CloudNative-environment (they run OpenShift on BareMetal-machines).
The end-customer sees a lot of – let’s call it that way – potential in the collaboration with the ops-vendor. Or to put it in clear words: There is no transparency, no knowledge-sharing, no automation, no versioning and no proper mindset in place at the ops-vendors teams. As a result, a lot of the infrastructure is somewhat running, but not even close to the standards, the end-customers expects. Rightfully expects, in my opinion.
This was the situation when Cloudibility experts were hired to analyze and fix errors in the environment, the software, and the collaboration. We proposed an agile DevOps-based interaction- and operations-model to our customer (the ops-vendor) and discussed chances for such a model with the end-customer as well. Both parties appeared to be very in close in their expressed opinions – yes, they all want to switch to a DevOps-approach.
One week later…
One week later, one of the two companies still sticks to that opinion, the other one does not.
The ops-vendor changed his mind since we perhaps were too successful and identified as well as fixed a lot of improvement-opportunities. He simply does not see the need to change his processes (which lead to the situation we came in) and his way of executing anymore, since the fixes are actually working and the customer appears to be satisfied.
DevOps now is to be executed by his customer, he will be participating in this „a little bit“ (quote), but only „as long as it fits into our processes and does not imply ourselves to change our approach“ (another quote). Plus: „We want to keep our internal approaches a secret, the customer does not need to know about them or can even expect us to adjust to his way of interacting – he is the customer, we are <Company-Name ommitted>“ (quote). And: „Don’t forget, we are not even required to have a logging and a monitoring in place, by the terms of the contract that customer signed“ (quote).
What could possibly go wrong with such a mindset?
Frankly, we expected this behavior from this customer, but it is frustrating nonetheless since they offer a CloudNative and Enterprise-ready environment and operations-model to their customers – and simply can not deliver due to having the wrong mindset and approaches, which gets only visible once a customer signed the contract (we have seen this several times with this customer). They execute IT the old-fashioned ways, having SDM (Service Delivery Managers), steep hierarchies and ITIL-processes in place, instead of DevOps-managers, experts, and an agile operations approach.
So, be careful when someone tells you about executing DevOps. Chances are, all they want is to get their backs covered and they will refuse DevOps as soon as it would imply to change something in their own processes and approaches.
Are we stopping to help such customers and to propose modern, lightweight, cost-efficient and adoptable processes and approaches?
And we will – of course! – successfully end our mission in that specific project as well. But it is kind of sad to see a chance for establishing better approaches and better processes pass by just because one of the involved parties does not want to change anything on their side.
Sad from a mindset’s perspective. From a commercial perspective, it is not: We are happily there to help with the next failing project. Again.
Yeah, we want to do DevOps. We understand this is a critical thing, it is important for succeeding in our Dev- and Ops-projects. We can’t do without.
…we need to get the whole picture first. We need to have a fully featured DevOps-concept. We need some infrastructure. We need top clarify with our Stakeholders.
You know what?
These are only excuses for not doing DevOps.
The best approach to DevOps is by simply starting it. Forget about that fully featured DevOps-concept to be discussed throughout all hierarchy levels of your organization. Forget about providing the perfect infrastructure. Forget about everything – just start it!
DevOps is a process and a mindset. It is an approach to collaboration, to transparency and to knowledge sharing. Yes, it always can be done better, more aligned, deeper integrated, etc.
But nonetheless: The best way to start doing DevOps is to simply start doing it.
I will never ever set up and maintain my environment by hand again!
I will never ever set up and maintain my environment by hand again!
I will never ever set up and maintain my environment by hand again!
If you ignore this advice, you might be ending like a lot of projects I’ve seen in the last years: Unmanageable, unstable, unpredictable and basically unreliable.
Use a tool
If you ever happen to set up a something in your environment, learn about tools like Ansible and perhaps Terraform first. Provision your machines and VMs with these tools, roll out your environment using these tools and version the scripts in a repository.
Do not, ever, later on, do changes or install updates on your environment by hand! Again, use Ansible or a similar tool, to roll out and install updates and components.
The key to automatization is using tools like GIT extensively. Every single configuration file, every single automatization script needs to be put under version control. Every iteration, every change, needs to be versioned as well. Get rid of your local script repositories, keep things in a central, safe place. Share the scripts and configurations, and don’t only document them in your ticketing tool!
Do not use SSH
Of course, SSH is used when working with Ansible or other automatization tools. But you, or any of your team members, should not use it. Using SSH to do tasks on a machine is by definition a manual process, something which has to be avoided! So, forget about SSH as a tool for manually managing infrastructures, configurations, and machines. Script your changes, test your changes, roll out your changes or roll them back – all using Ansible (or other similar tools) and version those scripts as well.
Automatization is key, the tool is not
Don’t feel comfortable with Ansible? Not an issue, use Chef or Puppet or any other automatization framework instead! Don’t want to learn about Terraform? Then go the native route using AWS-CLI or Azure-CLI instead. GIT sucks? Use SVN or CVS or Mercurial!
Regardless of the tool: You need to get the right mindset, and you need to get it, before starting any work! It never worked (and never will work) bringing in automatization and tools later on. You simply won’t be able to consolidate all different configurations without any bigger effort. It’s not gonna work!
Be a developer
Yeah, I know. You are not a developer. You are an administrator. You don’t program things. You don’t write nasty code. You are the specialist, the surgeon.
Well, no. You are a fool if you happen to think so.
You need to think like a developer thinks: Laziness over repetition, scripts over manual approaches, versioning over file-share-based storage. A developer – and believe me, I am one these guys – has a very simple approach: Every repetition of any kind of functionality to be implemented, is basically a wast of time.
A developer tries to write specific code only once, he organizes code in libraries for reusability. He refuses to do things a second time if he could reuse existing code or a library.
Adopt this kind of thinking! Express everything in scripts. Version these scripts. Create your own library of scripts and share it with your fellow colleagues!
Stay in control
I get often asked: What and when do I need to automatize? The answer is simple: Everything, anytime. The moment you SSH into a machine and do any kind of change there, you have lost control. Even if you are unsure about a configurational change being the proper solution to an issue, use a script.
Did I say „Even“? Especially then!
Using an automatization framework, you can roll back the change or set up an environment into a well-known state, allowing you to safely perform changes, test the outcomes and understand the consequences. Since you have versioned everything, you can always revert back to the last known version. Since you have everything in a shared, safe place, you can even lose your computer and your notes – and still remain operational.
And, in case it was not clear enough: This holds true for any kind of environment – Bare-Metal, virtualized, cloud, everything in between.
To stay in control, automatize and version. Everything!
I do often get this question when providing our specialist’s profiles to possible customers.
Well. No. We can’t. And we won’t.
Let me explain that bold statement a little bit. There are plenty of reasons for saying so. The most important one: We have some of the best specialists here at Cloudibility.
We love our experts.
We provide our bright minds with an environment which allows them to shine – a good work-life-balance, paid educational hours, etc. We do this, since we would have liked such an environment ourselves when being employed. But it was not possible due to calculation issues. Which forced us to quit our jobs.
Therefore, we want to pay our experts reasonable salaries and want them to gain even more knowledge. We want them to stay motivated and hunt for solutions instead of billed hours. We are investing a lot into our experts, and we are very happy and proud to do so. We know, that you deserve the best expert and the best approach for your project.
But it’s still expensive, though.
Do you know how much it costs, to do without good experts? Or to simply opt for the cheaper alternative since it is … cheaper? Can you really afford the second best or a somehow okayish solution? Just for the sake of saving some bucks forehand?
Our experts are worth their money.
They are experts and bright minds, they are able to solve problems, they are able to think outside the box. They save you time and money and nerves, they bring knowledge and experience into your projects.
So, no. We won’t do special prices.
We already have the best prices in the market since we have the best experts and the brightest minds here at Cloudibility.
Because you shut down your VMs at night. Automatically.
Because you have a Jenkins-installation.
Because you are moving to a cloud environment.
Because you have set up a „DevOps“-team.
Because you have a lot of meetings with stakeholders.
Because you want a Development team to run a software since the approach is often described as „You build it, you run it“.
Because you know about this nifty image on top.
Turns out: No.
You don’t do DevOps.
You just shut down your VMs at night, you just happen to have a Jenkins-installation, you’re just moving to a cloud environment, etc.
But this is not DevOps. At least not in the sense we at Cloudibility understand it and explain it to our customers and set it up with them. To us, DevOps is not about any specific technology or setting up a team.
DevOps is a mindset.
It is an approach to thinking about, developing and running software collaboratively. It is about the way you interact from the start to the end of a project with each other. It involves getting rid of this „throwing over the fence“ mentality. It involves a process for collecting and maintaining knowledge in an ever-changing team and agile approaches to development and operations. It is about the way a team is set up and how it evolves, it is about the way we set up and execute operational processes. DevOps even is a way to organize collaboration in a whole company.
So, DevOps is way more than putting Dev and Ops on the same table. Or than moving into cloud environments. Or than being agile. Way more.
In the following months and weeks, I will give you insights into our approach to DevOps. I will give you some tips and hints. I will help you to see the whole picture. I will do this on a per-issue and per-aspect base, and it will be a loose series of articles.
I’ve never seen a team being that committed. That focused. That inquisitive. That eager to learn. That funny. That great.
Team, I am so proud of working with you!
Next week, we will start our expansion. I mean: We will really start it. Three new Team-Members at the beginning of the month. Two more in the middle. One more beginning of August. One more at mid of August. One, two, three perhaps beginning of September.
But: I am not afraid. I know, that we already have a great team and we will continue to be one. We looked carefully at the new colleagues‘ character and found them to match the team in culture and commitment.
Sandra, Ronny, Emily: Welcome!
You will be part of Cloudibility, you will shape us and you will grow with us. And I am very confident to count you in next time when I write about the:
Just a short fact: As of now, we have around 70% of women and only 30% of men here at Cloudibility.
Of course, this is just a snapshot since the figures might change, as around 70-80% of all employees in IT are men – but it also depicts a fact: We are open to everyone.
We want to have the bright minds, not the bold muscles (although we do not have a problem with muscles at all – I personally try to convert some fat into some muscles for years without success at all, but this is a different story…).
We have established (and we can not live without it) a culture without glass ceilings, we want all of our stuff to have the same chances and opportunities. We want all of our stuff to overtake us in terms of knowledge and experience, we want all of them to stand by their opinions, discuss them, change them, renew them and grow together. We have no issues in being challenged and we want all of our stuff to bring everything on the table – as long as it is done with respect and openness to the other side’s arguments.
There are some things, we do not tolerate, though: Domination patterns, for example. Or this elbow-thing. Or ignoring a „Stop!“, regardless of the issue. Or when someone simply decides instead of explaining. Or the good old „we always did it that way“ approach.