Aggressive DevOps

A few myths about DevOps

2014/10/24 DevOps, Operations, Xi Group Ltd. 1 comment , , , ,

It has been more than 2 years, since Xi Group Ltd. ventured to provide DevOps services. We worked on multiple projects, for established companies and startups. In that period, we experienced multiple misconceptions about what DevOps actually is or what should it be. Here are the ones we find most common.

Myth: Developers can do Ops

People, lacking operational background, no matter the position or title, will fail to become DevOps Engineers. In the same way in which one can not build a house without proper foundation, becoming DevOps Engineer without operational expertise is futile. In fact, we experience the opposite to be true: Developers are generally bad at Ops. And there are complex reasons for this. Education, work experience and the separation of roles in most enterprises are mostly geared towards deep specialization. In contrast, the DevOps Engineer has to be able to fill-in for a lot of roles: sometimes system architect, sometimes build engineer, monitoring specialist or even quality assurance specialist. Hence to be a good DevOps candidate you need exposure to a lot of technologies and activities. Operation background is a must. Development background helps.

Myth: Sysadmins are obsolete

Saying that system administrator or system engineers are obsolete is like saying that Unix is obsolete. Of course it is … in its original form. But it has been evolving for more than 40 years now. And it is the same with the System Administrator role. DevOps is not revolutionary step. It is evolutional development of the role based on the changes in the environment. Adaptation is required. Culture is changing, the Cloud is mainstream technology today. In fact, many of those that were System Administrators before the DevOps movement emerged were doing ‘DevOps’ in one way or another. And those are the people you want to be on your DevOps team today. Without being disrespectful, we’d prefer forty-something-year-old Unix sysadmin that knows SH and “some TCL, Perl, Expect” to any monkey-patching, “Ruby/node.js’s the deal” developer!

Myth: DevOps can not be outsourced / off-shored

This a matter of organizational structure and culture. In the same way Operations or Development can be outsourced or off-shored, DevOps can be. Communication is key. Presence is important. But technology allows us to have it all without being physically in the same room. In our experience it boils down to company / team culture. If you have the right ingredients in the mix it will not matter where the guy that manages your 1000+ instances Hadoop cluster physically is. He may be in your conference room and if you ignore his input stuff will break. He may be hundred of miles away and if team cohesion is strong product will be deployed.

Myth: You need DevOps only if you use Cloud

Another common misconception. Although DevOps movement gained momentum with the increasing popularity of the Cloud those are not in a causal relation. You can apply most of the DevOps culture and principles without using cloud technologies. Any significantly complex system can benefit from them. The Cloud created specific challenges (elasticity!) that DevOps practices can meet, but the same or similar challenges are found in most distributed systems anyway. DevOps is first cultural, then technological.

Myth: DevOps is supplementary activity

This is common incarnation of the “we can add it later” mentality. In the industry, one will meet a lot of those. We get the “We can add it later” reply for requests about Security, Performance, Maintainability, etc. However, much like Security, if you’re committed to actually do DevOps – do it from the start. Include DevOps-aware participants in all phases of the Software Development Lifecycle (SDLC). Treating DevOps as supplementary activity will bring only sub-optimal results, which in turn translates into increased operational or development cost.

If you are committed to implement DevOps, don’t believe any of the myths mentioned above. Define your goals and create the environment to implement them successfully! After all, DevOps is about better/faster/smarter execution …

The “Aggressive DevOps” Concept

2014/06/09 DevOps, theCloud, Xi Group Ltd. , , , ,

Xi Group Ltd. is involved in operational and DevOps activities for several years now and during that period several lessons became clear:

  • Not all architectures are cloud-friendly;
  • You need to know your data flows to benefit from the cloud services;
  • Many companies use theCloud only as easy provisioning technology;
  • Elasticity is hard to implement!

Lets leave the former two for another blog posts and concentrate on the the latter.

Many companies use theCloud only as easy provisioning technology

… and it is natural when you come from a static infrastructure world. However, this is not what theCloud is all about. Yes, you can use it for that, in same way you can use a steam roller to iron your shirts. Yes, there are other arguments why you may want to put something in theCloud, but just copying your infrastructure in theCloud and stating that this makes it resilient of fault-tolerant is plain stupid! AWS had outages, by extension Heroku had outages, Joyent had outages … you need to design for reliability to achieve reliability. “Easy provisioning” is good, but if it’s your main reason, stick with the physical infrastructure. It is probably cheaper in the long run. However, if you want to fully utilize this technology, keep reading!

Elasticity is hard to implement!

Many workloads are elastic in nature, from website traffic to batch processing. The elastic nature of the workload may look like a sine wave or like a spike, the logic is the same. You’d generally want to adjust the amount of resources you allocate so that it’s enough to cover the workload, but also minimize the price you pay. And for any practical problem it is damn hard to do so! Why?! We identified the following reasons:

  • Software environment is chaotic;
  • Software deployment is non-trivial task;
  • Tightly-coupled architectures;
  • Data store functionality misaligned with the nature of the data;
  • Data stores inherently rigid;
  • Lack of operational information;
  • Monitoring wrong resources;
  • Monitoring returns unusable information;
  • There is no ‘control plane’ in the system;
  • ‘Control plane’ depends on infrastructure stability;
  • … and many, many others …

Those helped identify the following “pillars” as the foundation for large-scale elastic deployments:

  1. Build & Deployment Automation
  2. Full-stack Application Monitoring
  3. Data-driven Control Plane

Aggressive DevOps is the process of implementing the pillars in a business environment!

It is our company mission to develop and provide the tooling, the services and the know-how for others to implement large-elastic deployments.

Over a series of blog posts we will go into further details about each of Aggressive DevOps pillars. Stay tuned!

New Beginnings

2014/05/31 Xi Group Ltd. No comments , , , ,

Hello Dear Reader,

My name is Ivo Vachkov and I, with the help of my colleagues, run Xi Group Ltd. We are a small company devoted to high quality Operations / DevOps services with Cloud emphasis.

We build and run applications in the Cloud. And I’d like to share some of the lessons learned and some of the interesting stories from the Ops trenches.

I hope you’ll enjoy the tales to come. Learn from them and avoid our mistakes! Maybe even we’ll work together one day …

Ivo Vachkov
Xi Group Ltd.