Puppet - GraphViz and showing Dependency Cycles

can run puppet with --graph  and it'll make a dot file.

i think it automatically makes a dot file if it detects a dependency cycle.. 

1) apt-get install graphviz 

2) dot -Tpng somePuppetGraph.dot > output.png

___________________________________________________________ 

 


 _______________________________________________________________________

References:

https://docs.puppetlabs.com/guides/faq.html#how-do-i-use-puppets-graphing-support 

http://stackoverflow.com/questions/1494492/graphviz-how-to-go-from-dot-to-a-graph

Linux - A to Z of linux commands

http://ss64.com/bash/

Puppet - Containment

INCLUDE  !=  CONTAINMENT

Containment of CLASSES and RESOURCES are treated differently.

Classes never contain the classes they include. 

Classes contain the resources they declare.
Defined type instances contain the resources they declare.

The purpose of containment, in general, is to let you control where and when certain parts of your Puppet code are executed.

The anchor pattern is a MUST for all modules which may be used with Puppet 2.6.x or Puppet Enterprise 1.x.    A module author SHOULD contain the declared classes by declaring a begin and an end anchor, then relating the declared classes to these anchors.

Paraphrasing Again:
A class contains all of its resources. This means any relationships formed with the class as a whole will be extended to every resource in the class.
Classes can also contain other classes, but you must manually specify that a class should be contained.
Then there is another thing called require that I am not talking about.
Also not talking about contain which is relatively new.

http://blog.mayflower.de/4573-The-Puppet-Anchor-Pattern-in-Practice.html
http://puppetlabs.com/blog/class-containment-puppet
http://projects.puppetlabs.com/projects/puppet/wiki/Anchor_Pattern


https://docs.puppetlabs.com/puppet/latest/reference/lang_containment.html

AWS - S3

So we use Duply to back up LVM snapshots of our mongo database.
And Duply is configured to send the backups to a bucket on S3.
Well.... occasionally we'd delete a Mongo VM and then we'd end up with orphan data in S3.
Stumbled around and found a tool to view S3 easily and realized we had quite a buncha junk in there.
So wrote some python to list the files in S3 and then to delete the junk.


 CyberDuck:
http://cyberduck.ch/

Boto.S3:
http://boto.readthedocs.org/en/latest/s3_tut.html

Linux - Disown

scp -r sbuttrick@blah.somewhere.com:/home/sbuttrick/jenkins/jobs .
CTRL + Z       //to stop
bg                    //to background
jobs                 //to see job number
disown %n      //where n is the job number.. and yes.. it requires the weird % thing in there



AND if you had planned ahead in the first place (crazy huh?) then you should have used "nohup"

Linux - Packages

Why does what version of a package get installed on Ubuntu?

apt-get install jenkins
(installs jenkins)

dpkg -l |grep jenk
(to list what version is installed)
ii  jenkins     1.424.6+dfsg-1ubuntu0.2     Continuous Integration and Job Scheduling Server

Then head over to http://packages.ubuntu.com/
and pick your version of Ubuntu and do a SEARCH.

And hopefully what comes back is the same as what you had installed.

ALSO..........
And it gets an upstart script put in place in  /etc/init/jenkins
which reads from /etc/default/jenkins

AWS - Disk IO

Goal:

• Test a few different instance sizes and storage configurations to gauge their performance relative to each other.

 Constraints:

• Our provider will not support EBS Optimized instances.

Other things to keep in mind:

• Critical to test multiple instances of same flavor (link)
• Its important to test the Ephemeral/Instance storage because its physically attached to the machine (link) and that probably offers performance benefits.  *Of course if we use instance storage we need to  keep data safe by using a replication strategy across multiple instances and storing backups in Amazon S3.
• R3.large & Family:  These instances are the sweet spot for your MongoDB instances. They have the right balance of memory and compute power. They are good candidates to run your larger MongoDB server. MongoDB is mainly a memory game – the more memory you supply the better it works and these instance types offer the most memory. The previous generation of these instances used to be called M2. If your MongoDB server is still getting disk bound then I would consider the High IO instances.  (link)

So here are the combinations I'd like to test:


m1.large

• m1.large with EBS storage
• m1.large with Ephemeral/Instance storage (412gb spindisk)
• cost:  $.175 / hour  (link)
• "moderate" network performance  (link)
• *optional* ebs optimized  (link)
• regular spinning disk  (link)

m3.large

• m3.large with EBS storage
• m3.large with Ephemeral/Instance storage  (32gb ssd) 
• cost: $.140 / hour  (link)
• "moderate" network performance  (link)
• *NOT* ebs optimized  (link)
• SSD disk  (link)

m3.xl

• m3.XL with EBS storage
• m3.large with Ephemeral/Instance storage  (80gb ssd)
• cost: $.280 / hour
• "high" network performance
• SSD disk

Measurements with FIO
(doing measurements with fio)
(interpreting link)

Results:

1. m1.large & ephemeral & read
2. m1.large & ephemeral & write
3. m1.large & ebs & read
4. m1.large & ebs & write
5. m3.large & ephemeral & read
6. m3.large & ephemeral & write
7. m3.large & ebs & read
8. m3.large & ebs & write
9. m3.xl & ephemeral & read
10. m3.xl & ephemeral & write
11. m3.xl & ebs & read
12. m3.xl & ebs & write

Appendix:

ephemeral/instance storage types

1. spinning disk
2. ssd disk

network performance values

1. low
2. moderate
3. high
4. 10gigabit

Linux - lsof

list open files.

awesome for figuring out what files a program is using...
maybe to determine where logs are being written...
maybe to show which data files mongo is using...

lots of use cases

Linux - mdadm adventure

Saturday night we were alerted to high load on various mongo-prod-boxes.

I ssh-ed into the machine and found a few processes that I was unfamiliar with consuming about 10% cpu.

They were  md0_raid10   and   md0_resync .

Googled around a bit.... seems to be something to do with the RAID array configured for us.

 

Used tooling to run "sudo mdadm -D /dev/md0" across various machines and found a good chunk of them with "Check Status : 93% complete" or similar percentages.

Seems that some event happened cuz these all flaked about the same time.

 

 still not sure what the root was.....

 

 

 

Puppet - setting up a Jenkins Server

Used this module to stand up a Jenkins Master and some Slaves:
https://forge.puppetlabs.com/rtyler/jenkins


Eventually hope to use this python API to attach the slaves to the master after their creation:
https://pypi.python.org/pypi/jenkinsapi

Unfortunately the Jenkins-Swarm plugin doesn't work on AWS cuz they block UDP Broadcasts.

You can also poke at the Jenkins API through your browser:
 http://someJenkinsHost:8080/api/json?pretty&depth=3


and for some reason our legacy Jenkins server has like 90 plugins... so we'll probly want to weed that down quite a bit.

Linux - making sense of all those attached devices

-----------------------------------------------------------------------------------------------------
intmdbshard-modb-s11-0001:~$ lsblk

NAME                       MAJ:MIN RM   SIZE RO TYPE   MOUNTPOINT

xvda1                      202:1    0     8G  0 disk   /
xvdb                       202:16   0    30G  0 disk
xvdf                       202:80   0   100G  0 disk
└─md0                        9:0    0 199.9G  0 raid10
  └─data (dm-0)            252:0    0 199.9G  0 crypt
    └─datavg-datalv (dm-1) 252:1    0 159.9G  0 lvm    /data
xvdg                       202:96   0   100G  0 disk
└─md0                        9:0    0 199.9G  0 raid10
  └─data (dm-0)            252:0    0 199.9G  0 crypt
    └─datavg-datalv (dm-1) 252:1    0 159.9G  0 lvm    /data
xvdh                       202:112  0   100G  0 disk
└─md0                        9:0    0 199.9G  0 raid10
  └─data (dm-0)            252:0    0 199.9G  0 crypt
    └─datavg-datalv (dm-1) 252:1    0 159.9G  0 lvm    /data
xvdi                       202:128  0   100G  0 disk
└─md0                        9:0    0 199.9G  0 raid10
  └─data (dm-0)            252:0    0 199.9G  0 crypt
    └─datavg-datalv (dm-1) 252:1    0 159.9G  0 lvm    /data

-------------------------------------------------------------------------------------------------

intmdbshard-modb-s11-0001:/mnt$ df -h
Filesystem                            Size     Used      Avail     Use%       Mounted on
/dev/xvda1                            7.9G     7.3G     207M     98%         /
udev                                      3.7G     8.0K      3.7G      1%          /dev
tmpfs                                     1.5G     264K     1.5G     1%           /run
none                                      5.0M        0         5.0M     0%          /run/lock
none                                      3.7G        0          3.7G      0%          /run/shm
/dev/mapper/datavg-datalv  158G      18G      133G    12%          /data
/dev/xvdb                               30G     173M      28G     1%            /testingMount


-------------------------------------------------------------------------------------------------

src:
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-using-volumes.html

Puppet - Dependency Graphs

run puppet with the  --graph  option and Puppet will create a DOT file.
View the DOT file with Omnigraffle or even LucidCharts.

https://docs.puppetlabs.com/guides/faq.html#how-do-i-use-puppets-graphing-support
http://bitfieldconsulting.com/puppet-dependency-graphs