wget - downloading website recursively

So there is this humor & stuff website I like and I wanted to have a push-button approach to getting the posts of the day so I could browse it on the train where the wifi sucks.

Came up with this:

wget --domains=thechive.com,thechive.files.wordpress.com -r -l 1 --restrict-file-names=windows -E -H -k -K -p -e robots=off 'http://thechive.com/'

So all the images get grabbed ..but they have some wonky suffix.
So threw together a shell script to clean em up:

for f in *; do
    z=$(echo "$f" | cut -d"@" -f 1)
    echo "$z"
    mv "$f" "$z"
done

Docker and Beaker (Puppet Testing)

Sooo... we've historically been very bad about testing our puppet stuff.
We don't really do module level testing.
We manually do end-to-end testing.  I guess its been 'good enough' so far...although tedious.

Anyways, I thought we should chart some new territory and attempt to use Beaker to some degree.

Beaker Wiki: link

Anyways...turns out they support Docker.
I thought, yay, finally an excuse to play with Docker a bit.

Turns out it was a pain in the ass to setup though.

1. Get the Ubuntu 14 vagrant box:
   https://atlas.hashicorp.com/ubuntu/boxes/trusty64
    
2. SSH into that box
    
3. Install Beaker on that box
   https://github.com/puppetlabs/beaker/wiki/Beaker-Installation#installing-beaker
    
4. Install Docker on that box
   https://docs.docker.com/installation/ubuntulinux/
    
5. Create a 'beaker hosts config file' on that box
   example file:  "beaker_hosts.cfg"
   
   HOSTS:
     ubuntu-12-10:
       platform: ubuntu-12.10-x64
       image: modified_ubuntu
       hypervisor: docker
   CONFIG:
     type: foss

6. Do a simple 'docker run' to get docker to download the Ubuntu Container Image:
   sudo docker run ubuntu:12.10 /bin/echo 'shit'
    
7. Modify the Ubuntu Container Image by installing 'ssh' onto it:
   sudo docker run ubuntu:12.10 apt-get install -y ssh
    
8. Find the 'container id' after the previous step finishes.
   sudo docker ps -l
    
9. Commit the changes to the container
   sudo docker commit someFriggingUglyContainerId modified_ubuntu
    
10. Confirm that 'ssh' is installed:
    a) open an interactive session with the new container thingy:
    sudo docker run -i -t modified_ubuntu /bin/bash
    b) start the 'ssh' service on the new container thingy:
    service ssh start
    c) quit the new container thingy:
    exit
     
11. Run beaker on the VM !!
    sudo beaker --log-level debug --hosts beaker_hosts.cfg
    
12. Happy Dance.

The following images MAY have differently named files/settings... so just use your brain:

MongoDB - Authentication

system.users collection  associates a user+database with 1+ roles
{http://docs.mongodb.org/v2.6/reference/system-roles-collection/}

system.roles collection  associates a role+database with 1+ privileges and/or 1+ roles to inherit
{http://docs.mongodb.org/v2.6/reference/system-users-collection/}

• user
  {http://docs.mongodb.org/v2.6/reference/command/#user-management-commands}
  MongoDB scopes a user to the database in which the user is created.

• role
  {http://docs.mongodb.org/v2.6/reference/built-in-roles/} 
  {http://docs.mongodb.org/v2.6/reference/system-roles-collection/} 
  {http://docs.mongodb.org/v2.6/reference/command/#role-management-commands}
  role is a collection of privileges (privilege is a resource and 1+ actions)
  role applies to database on which its defined
  MongoDB stores all role information in the admin.system.roles collection in the admin database.

1. Database User Roles
   {http://docs.mongodb.org/v2.6/reference/built-in-roles/#database-user-roles}
   {read, readWrite}
2. Database Admin Roles 
   {http://docs.mongodb.org/v2.6/reference/built-in-roles/#database-administration-roles}    {dbAdmin, dbOwner, userAdmin}
3. Cluster Admin Roles 
   {http://docs.mongodb.org/v2.6/reference/built-in-roles/#cluster-administration-roles}    {clusterAdmin, clusterManager}
4. Backup & Restore Roles 
   {http://docs.mongodb.org/v2.6/reference/built-in-roles/#backup-and-restoration-roles}    {backup, restore}
5. All-Database Roles 
   {http://docs.mongodb.org/v2.6/reference/built-in-roles/#all-database-roles}{readAnyDatabase, readWriteAnyDatabase, userAdminAnyDatabase, dbAdminAnyDatabase}
6. SuperUser Roles
   {http://docs.mongodb.org/v2.6/reference/built-in-roles/#superuser-roles}
   {root, userAdminAnyDatabase, and (dbOwner or userAdmin)when scoped to admin database}

• privilege 
  {http://docs.mongodb.org/v2.6/reference/privilege-actions/}
  privilege is a resource and 1+ actions

• resource 
  {http://docs.mongodb.org/v2.6/reference/resource-document/}
  {Database, Collection, Cluster}

• action 
  {http://docs.mongodb.org/v2.6/reference/privilege-actions/}
  define the operations a user can perform on a resource
1. Query & Write actions 
   {http://docs.mongodb.org/v2.6/reference/privilege-actions/#query-and-write-actions}   
   {find, insert, remove, update}
2. Database Management actions 
   {http://docs.mongodb.org/v2.6/reference/privilege-actions/#database-management-actions}
   {changeCustomData, changeOwnCustomData, createCollection, createIndex, createRole, createUser, enableProfiler, killCursors, unlock, etc etc}
3.   Deployment Management actions 
   {http://docs.mongodb.org/v2.6/reference/privilege-actions/#deployment-management-actions}
   {authSchemaUpgrade, cleanupOrphaned, cpuProfiler, inprog, killop, planCacheRead, storageDetails, etc etc}
4. Replication actions 
   {http://docs.mongodb.org/v2.6/reference/privilege-actions/#replication-actions}   
   {appendOplogNote, replSetConfigure, replSetGetStatus, resync, etc etc}
5. Sharding actions 
   {http://docs.mongodb.org/v2.6/reference/privilege-actions/#sharding-actions}
   {addShard, enableSharding, flushRouterConfig, listShards, moveChunk, shardingState, etc etc} 
6. Server Admin actions 
   {http://docs.mongodb.org/v2.6/reference/privilege-actions/#server-administration-actions}
   {closeAllDatabases, collMod, compact, convertToCapped, dropDatabase, dropIndex, hostInfo, logRotate, repairDatabase, shutdown, touch, etc etc} 
7. Diagnostic actions 
   {http://docs.mongodb.org/v2.6/reference/privilege-actions/#diagnostic-actions}
   {collStats, connPoolStats, cursorInfo, dbStats, getLog, indexStats, listDatabases, netstat, serverStatus, top, }  
8. Internal actions 
   {http://docs.mongodb.org/v2.6/reference/privilege-actions/#internal-actions}
   {anyAction, internal}

Cape Cod 2015 - Part 3

 

 

 

 

 

 

 

 

 

Cape Cod 2015 - Part 2

Cape Cod 2015 - Part 1

 

Docker - finding canned images to use

https://registry.hub.docker.com/

Images for Ubuntu,  Redis, Wordpress, Mongo, NGinX, NodeJS, etc etc