CASSANDRA AND AZURE RESOURCE MANAGERHow it works and how to get it moving

CARLOS ROLOFEBRUARY 2016

WHO AM I?• Cassandra Consultant for Pythian• I’m all about Distributed Systems• Certified Datastax Architect• Cassandra MVP• Programming since 1997• Cassandra DBA since 2001• Previous experience:

– INESC/ID– DRI– Portugal Telecom– Leaseweb CDN

ABOUT PYTHIAN• 18 Years of Data infrastructure

management consulting

• 200+ Top brands

• 6000+ databases undermanagement

• Over 400 DBA’s, in 29 countries

• Top 5% of DBA work force,– 9 Oracle ACE’s, – 2 Microsoft MVP’s,– 1 Cassandra MVP

• Oracle, Microsoft, MySQL,Datastax partners, Netezza,Hadoop and MongoDB plusUNIX Sysadmin and Oracle apps

© 2014 Pythian Confidential4

AGENDA• Azure• Azure and Cassandra• Azure Templates• Automation• Takeaways• Q&A

© 2014 Pythian Confidential5

AZURE• Microsoft Cloud Service• 53 different services• 27 Locations (22 Active)• CLI offerings for Linux, Mac, Windows

© 2014 Pythian Confidential6

AZURE RESOURCE MANAGER• Azure Resource Manager (ARM) is a

“package”– You assign resources to your Resource Group– You can track all the resources from a single

point, including:• Billing• Monitoring• Access Control• Template based provisioning

© 2014 Pythian Confidential7

AZURE INTEGRATIONS• Windows: Powershell, Azure-Cli, Visual

Studio• Linux, Mac: Azure-Cli• + Web Interface• + Rest API

• Level of integration– Powershell & Visual Studio >>> Azure-Cli

© 2014 Pythian Confidential8

AZURE AND ME• It has been an uphill fight!

– But I’m getting more and more out of it, and I like it!

• But it was only one way to do things…– Then “classic” and “arm”

• Powershell was good…– Stuff broke between 0.8.x and 1.0.x

• Let’s use cli then!

© 2014 Pythian Confidential9

AZURE AND ME(2)• Documentation,

Features…

© 2014 Pythian Confidential10

AZURE AND ME(3)• Fully migrated from AWS• 3 Cassandra Clusters in it• Automated* deployments

– ARM and Classic are just a one-liner way.• And it defaults to Classic!

– Deleting resource groups sometimes gets weird• Some UUID is not allowing you to do whatever• Can’t find the UUID anywhere

© 2014 Pythian Confidential11

AZURE AND CASSANDRA• Why?

1. Instance sizing: D4, D5, G2 and G32. Marketplace: Few clicks N node cluster (DSE)!3. Integrations4. Lots of Locations5. Latency, storage latency!6. Network!

© 2014 Pythian Confidential12

AZURE AND CASSANDRA (2)• And the downsides?

– Storage has a lot of stuff going on,– Throughput… (YMMV)– The web portal takes time to update

© 2014 Pythian Confidential13

AZURE AND CASSANDRA (3)• 8 clicks from 0 to “how-many-nodes-you-

need”!• Took me 4 or 5 min to go through all that!

© 2014 Pythian Confidential14

AZURE TEMPLATES• This is where automation gets in!

– Painfull to learn, but…– … lots of examples available (github)– Visual Studio makes wonders here!

© 2014 Pythian Confidential15

AZURE TEMPLATES (2)Seriously, if on windows, use Visual Studio!

© 2014 Pythian Confidential16

AZURE TEMPLATES, THE GOOD• Easy to use• C# + Visual Studio + Templates• Github + azure.deploy.com + Templates• Lots of examples available!• https://feedback.azure.com

© 2014 Pythian Confidential17

AZURE TEMPLATES, THE BAD• Did I told you how painful it was?• And look at all the documentation I can’t

find!• All those functions…

© 2014 Pythian Confidential18

AZURE TEMPLATES, AND THE UGLY• Testing?

– The only way to test is to deploy…

© 2014 Pythian Confidential19

AUTOMATION• Linux, Mac

– Azure-cli• Windows

– Powershell– Azure-cli

• Azure Automation Service

© 2014 Pythian Confidential20

AUTOMATION (2)How do we make this get Cassandra automated?• “commandToExecute”• You can have your scripts in Azure

Storage, github or make them copied over to your VM

• It can be set to variables and things can get complex fast.– But it is the best way to do it!

© 2014 Pythian Confidential21

AUTOMATION (3)

Provision

• Configure VM’s• Deploy VM’s and remaining Infrastructure

Install

• Install OS Packages• Install Cassandra

Configure

• Set configuration• Start the service

© 2014 Pythian Confidential22

AUTOMATION GOTCHAS• SSH-Keys…

– Read the docs, in the past some weird problems appeared.

• Lots of resources,– For powershell!

• Getting handle on those “commandsToExecute”

• Read the logs!!

© 2014 Pythian Confidential23

TAKEAWAYS• You’ll need patience if you’re coming from

AWS– You will go through a lot of trial and error.– Documentation and examples are scarce

• Azure is moving really fast– Can’t remember the last day I didn’t have an

update!– And can break stuff.

© 2014 Pythian Confidential24

TAKEAWAYS (2)• Beware of storage!• Setting up domains• Lots of tools if you are on Windows.• You can deploy not knowing anything

about Cassandra!

Q & A!• Twitter: @cjrolo• LinkedIn: linkedin.com/carlosjuzarterolo• Blog: blog.pythian.com/carlosrolo