Virtual Private Networking in AWS

Been doing lots of VPN setup and configuration lately, especially inside of Amazon Web Service (AWS) Virtual Private Clouds (VPCs.)  They have a built-in VPN capability using IPSec but it generally seems specifically focused on device-to-device (D2D) configurations.  Depending on the need I have turned up StrongSwan and/or OpenVPN as a solution.

OpenVPN has an advantage of being able to do SSL VPN on 443 making it look exactly like HTTPS web traffic (effectively making it unbreakable by network administrators.)  Things like proxy-servers don’t even know you are creating a VPN tunnel.  However, on Windows OpenVPN client software has to be installed to use it.

StrongSwan is a IPSec VPN option that works well with existing P2P VPN systems.  The native Windows VPN tools work out of the box with a standard StrongSwan configuration (as long as your certs have been signed by a trusted CA.)  Performance is also very good.

So far, I really really like OpenVPN as once it is configured it works everywhere, regardless of network policy or ISP limitations.  Linux Network Manager has built in support for it making is very very easy to configure clients to use it as well.  That said, for IPSec configurations needing to connect to Windows Clients; StrongSwan has been my go-to solution.

Useful links follow:

Linux StrongSwan Server

Workstation StrongSwan Setup/Install Client

OpenVPN on Ubuntu

drift toward unparalleled catastrophe

My home configuration has two Planar 20″ monitors as my primary display.  They have worked fairly well with the exception that any sudden change in input signal seems to cause them to freak out and changed their sync levels to non-standard ranges.  Resetting them is the fix but Planar is kind enough to NOT mention how to do that in any of their documentation.  So, for the benifit of mankind here is the process for resetting a Planar PL2010MW to factory default>

Unplug the monitor.  Counting from the left, there are five buttons on the bottom of the monitor (the right most being the power button.)  Press the second and fourth buttons from the left and hold them down while plugging the monitor back in.  Count to five, and release.

Other models of Planar use the second and third buttons with variations of releasing immediately after plugging in; or waiting until the main power light turns green.  In addition, if you are using some versions of Linux you may have to restart X before you see the minor in your hardware setup.

All I see are tabs

Cloud:

  • BitTorrent Sync – Multiple source file syncing using bit torrent client.  Thank of it as headless Dropbox.
  • Own Cloud – Open Source personnel cloud solution. Includes things like data, music, contacts, calendar and can even be used by multiple clients. Even set it up on your own server.
  • SparkleShare – Dropbox like functionality on Linux, Mac, and Windows systems. Includes versioning as well.
  • Gmail Forwarder –  Correctly configure gmail when using your own smtp settings, domain, and email forwarder.
  • Github For Everything – Using Github to manage everything in your company; from your hiring process to your internal documentation.
  • Using Gnu StowStow is a open source tool for managing your dotfiles in a universal way across multiple machines.  This also simplifies the process of using a version control system to track your dotfile history.
  • Git-Annex – Headless, versions, unlimited, decentralized file syncronization for Unix systems.  Based on Git and includes a mobile app.  Possibly the best replacement for Dropbox available anywhere.

App Dev

  • Apple App Distribution – All 100 freggin pages of it.  Includes beta testing and is almost like developing software back in the 90’s.
  • Android App Distribution – Eight pages and you can even using Google Groups to manage your beta test groups.  Includes automatic updates.
  • Ruby Rack nginx – Very clean, very simple example of setting up a Ruby Rack nginx configuration.
  • Source Code Comments – A list of the most humorous source code comments people have read.
  • Testing Code, Simply – I love this post.  Best simplification of how/why TDD should be used.  The examples can even be modified to allow testing of things like Bash or VBA.

Bash & SSH

  • More cool bash commands and shortcuts.
  • Need a web server, how about a single line of bash.
    while true; do { echo -e 'HTTP/1.1 200 OK\r\n'; cat index.html; } | nc -l 8080; done
  • Resetting an unresponsive SSH session.
    newline ~.
  • Setting up SSH to use shared concurrent connections.  Came from a tutorial on speeding up Git, but useful everywhere.
  • More SSH Awesomeness – This was how I learned ssh-copy-id.  Lots of other amazing advice.
  • Improve SSH Key Security – Things to do AFTER you have placed a passphrase on your SSH keys.  You have done, at least, that?  Right??
  • Passing SSH commands in git clone – Stackoverflow reponse on how to configure .ssh/config options for specific hosts.  They, in tern, get picked up by git.

Go

  • Go Tutorial Exercises – I have really enjoyed GO lately.  C language power with a language actually built for multi-core processors in a network connected world.
  • Effective Go FAQ – Some really great tutorials and information for developers trying out a new language.
  • Hermans Go – Project Euler solutions written in Go.  Great example code for learning algorithms in Go.

Vim

  • Nouns & Version – Understanding the basic structure of how VIM works.
  • Colons are Bad – How to stop using colon commands in VIM.
  • YADR – Sample dot files, vimrc, git support and other useful vim tools.
  • Vim Bookmarks – How to use and manage bookmarks in Vim.
  • Yankstack – A plugin to give kill ring capabilities to the Vim.
  • Block Shift – Visual block shifting in Vim.  Tab and un-tabbing, spacing, and block selections are all covered.
  • Vim Adventures – Learn Vim while playing a video game.
  • Awesome Vimrc – That is the name its developer gave it, not mine.  Still, it is a pretty cool, VERY clean vimrc file that has a lot of good examples in it.  Also can be found on github.

Misc

  • Programming Books – List of freely available programming books.
  • Effing Package Manager – Create rpm, deb package building directly from gems and bundler.
  • View your Axciom Data – Axciom is one of the largest data brokers of personnel information on the planet.  This website allows you to see the data that Axciom has on you.  The downside?  They get to keep the data you have to submit to see the data they already have.
  • Large Distributed System – Advice from people who build Google.
  • Faynmen Lectures on Physics – Everything you have ever wanted to know about almost everything that we think we know.
  • Nginx Secure Configuration – Setting up and securing nginx with ssl.
  • Debugging Broken postinst on Debian – Basically the postinst file gets installed anyway, so you just need to edit it on the semi-installed machine and then run it again… until it is fixed.
  • Bruce Schneier’s Sept 2013 Cryptogram –  Read this if you want a better explanation of why you should be VERY VERY afraid of what the NSA and large internet companies are doing.  Some articles are very technical but others are surprisingly approachable to the lay person.

I am not a link bot… I hope

Once again my desktop has become to cluttered with links.  Here are some of the ones I have been using the last couple weeks.

Vim

  • Vim Cheet Sheet – A short list of useful Vim commands & short-cuts.
  • Vim copy and past commands – Setting blocks, yank, paste, cut, etc.. in vim
  • Vim word completion  – Found this more useful after binding it the completion command to the tab key (aka bash mode.)
  • Remove unwanted spaces – Because some “people” think using spaces instead of tabs is a good idea.
  • Accessing the System clipboard in Vim – Because Vim registers do not necessarily map to the OS clipboard.  The quick summary is that I would strongly recommend putting the following alias in your .bashrc if type “gvim” > /dev/null; then alias vim=”gvim -v”; fi then make sure you have gvim installed.
  • Using Vim Registers – Actually using the registered mentioned above.
  • Pasting in Visual mode – Using registers is great but not really useful if you keep having to switch back to command mode to use them.

DBus

Ruby

  • Singing with Sinatra Pt. 2 – Sinatra is a ultra simplified application server environment for Ruby.   Think Rails only about 1/10th its size.  This was the best of the tutorials I found for it.
  • Thin Server Production and static files – This little blurb was something I caught on StackOverflow and knew I would need for later as our production system is running into the same issue.
  • fpm (freggin package manager) – Tool for creating deb/rpm packages from lists of filesystem files.  Particularly useful for gem files (it even has it as an option.) I am in the process of moving over my existing ruby build scripts over to fpm.

Debian

  • Creating Meta Packages – Meta packages are simply empty deb packages that contain nothing but a list of dependancies.  This way you can create a batch of files to be installed for a given purpose (like installing KDE Desktop.)
  • equivs-control man page- Used in the creation of Meta packages
  • Binary Package building tutorial for Debian – The deb build package environment basically builds itself around have source for all software.  This is a problem for packaging non-open source programs that don’t provide a source.  This is a tutorial for how to do it.
  • Template Changes file – Debian apt repositories generally work with .changes files to actually publish their packages.  This is an example of a changes file for the package dpkg-ruby.
  • Create you own apt repository – Includes information on upload support (which uses changes files mentioned above.)
  • Creating a basic Ruby application structure – How to create you base dependencies, directory structure, and file-system layout for a base Ruby project.

Its appointed time for everthing

If you are a command line junkie, you really need to check out @climagic on twitter.  Some days are better than others but I am constantly amazed at what is possible in bash/csh.  That said, here are a couple commands I have needed recently, many will be worthless to anyone else but oh well:

  • ar vx mypackage.deb Unpackage a Debian binary install package. The result is actually three tar.gz files
  • dpkg -l  – List all installed Debian packges on a given system.
  • dpkg -c mypacakge.deb  –  List all files provided by the named Debian package.
  • hub pull-request -i 123 -b account/project:master -h account/project:branchtomerge  –  hub is a github utility that allows you to use some github functionality directly from the command line.  The preceding command will issue a pull request for branchtomerge into master and even tie the request to a given issue number (in this case issue #123.)
  • echo $(sha256sum $DEB | cut -f1 -d’ ‘) $(ls -l $DEB | cut -f5 -d’ ‘) $(basename $DEB)  –  This command creates the package hash structured named used INSIDE of Debian changes files.  Using the same command with (sha1sum|md5sum|sha256sum) will provided all three needed package id’s.  The reason this is useful is when you need to recreate a changes file without the original source package.  The rest of the file is fairly straight-forward but the signed package section has to absolutely precise. Also check out this link for more information.
  • asciiquarium  – OK, you might have to install this one first, but it is a full aquarium in ascii characters, including sharks that eat the fish.  Submarines, fishing hooks, and even the lock ness monster.
  • grc tail -f /var/log/maillog –  Note to self, I need to make an rpm for this package.  grc is a generic colorizer for other command line programs that don’t use color by default (like tail, traceroute, syslog, etc…)
  • isohybrid -h 64 -s 32 mycdimage.iso  –  Adds a simple filesystem layout to a standard iso image so it can be written to USB drive as well as an regular CD.  Really useful for building custom Linux CD/USB images.

Forms it never takes, places it can never be

So, after looking around for an answer today I finally found out where the Debian install CD stores its cd/usb boot menu configuration files.  While I have already had a great deal of experience editing grub.conf files by hand, this methodology simply doesn’t work on an “El Torito” Joliet CDROM image.  So Debian set-up their boot image (as part of the initial ram disk) inside of the /ISO/isolinux/ directory where ISO is the uncompressed version of the boot image.  Specifically you can configure things like:

  • The boot option timeout in /isolinux/isolinux.cfg
  • The background splash image in /isolinux./splash.png (640×480 on the default menu set-up)
  • Which sub-menu’s, options, boot methods, and GUI installs are available via the /isolinux/menu.cfg

Honestly, I may be the only person on the planet trying to figure this stuff out; but here it is for future reference or for anyone who wants to make their very own custom Debian install CD .

Building Debian For Fun and Profit

I needed to document the process I used to get a private Debian package repository with some custom applications.  Here is the process I went through.  You can find some reference links under my previous articles “Debian” heading.

Building

Basic steps to building setting up your personnel project to build Debian packages.

  1. Get most recent software version for your project (git pull, etc..)
  2. Rename project folder to include a default project number.  For example $mv myProject myProject-1.0
  3. In project directory run dh_make.  This will create a debian diectory with all the necessary files to build a deb package.
  4. Now would be a good time to edit your debian/control file and make any changes needed.
  5. If you project is a simple copy operation (say a php web application being installed onto an existing apache server) you will want to do the following:
    1. Create and edit a new file under debian/ named myProject.install (replace myProject with the name of your project.)
    2. Edit the file to specify the where you want the files copied to.  The format of the should be something like this (notice you can use wildcards:)

      myfolder/bin/* usr/bin
      src/etc/myproject.config etc/
      myfille usr/share/myfolder

    3. Make sure you debian/rules file looks something like this:

      #!/usr/bin/make -f

      %:
      dh $@

  6. Now, you should be able to build your package (and the changes & dsc files) by running dpkg-buildpackage.  The resulting packages will be in the directory one level up.

Hosting:

Hosting your own git repository can vary in complexity depending on which software you use to actually build the repository.  The easiest one I found to setup was mini-dinstall.  Start by installing mini-dinstall and apache2 via apt.  Apache is configured to start a basic server (all that we need) with the web page files hosted in /var/www.  Just clean out the www directory, add the mini-dinstall folders after the install:

rm -rf /var/www/*
mkdir -p /var/www/mini-dinstall/incoming

You will also need to create a configuration file for mini-dinstall to use when creating the package repository supporting files. Create/edit the file /etc/mini-dinstall.conf with something like the following:

[DEFAULT]
archivedir = /var/www
mail_to =
verify_sigs = false
architectures = amd64
archive_style = simple-subdir
generate_release = true
mail_on_success = false
release_codename = myreponame
release_description = My Repo Name  Hosting
release_label = myrepo
release_origin = myrepo

At this point mini-dinstall could be configured to run in server mode and watch for incoming packages, but using the utility below I have mine configured to run in batch mode every time I put new files on the hosted server.  This will cause mini-dinstall to create a debian package repository structure that can be accessed directly via apt.  Just add something like the following to your /etc/sources.list

deb http://yourserver.com unstable/abd64/

Deploying:

There is a really nice utility called dput that can be used for deploying software packages (and change files) to a hosting server.  The easiest way to get started is to install dput and then setup a configuration file.  Create a file in your home directory called .dput.cf (or globally in /etc/dput.cf) and add a deployment location like this:

[myservername]
fqdn                    = debian.myserverurl.com
method               = scp
incoming            = /var/www/mini-dinstall/incoming
login                    = root
post_upload_command = ssh root@debian.myserverurl.com mini-dinstall -b

That last line creates the repository using mini-dinstall mentioned above in section “hosting”.  This is particularly useful if you already share public keys with the remote system via ssh.  One you have set it up you can do deployment by typing:

dput -u myservername myproject.change

Where myproject.change is the file created above in “Building”.

Looking for Trouble

My goal in life (and this blog) is not to become a gigantic link-bot but I never seem to finish all the articles I want to save and return to… until I do. So here is the most recent reading life I have for June of 2013.

Other Technology

  • Build Your Own Google TV using Linux, Nodejs, Socket.io, Linux, and a RaspberryPi
  • arkOS your own personnel home cloud (without the NSA) on a $35 RaspberryPi.
  • TTYtter a command line based twitter client for Unix.  Can be run in disconnected mode (for a stand alone twitter “bot”), has some initial support for libnotify, and can even be scripted.
  • Using Git to backup $HOME One developers experience using Git as a home directory backup, tracking, and versioning system.  I am working on the same idea right now.
  • Configuring Keyboard Layouts on a per keyboard basis.  Particularly useful when F*$*(#NG Apple decides to move the Alt and Cmd keys from their 30 year old locations… but all your USB keyboards use the default locations.

Start-ups and Business

  • Startup Advice 95 pieces of advice Sam Altman has heard about creating, managing, and developing a startup.

Software Development

  • Shortcut Training Interval Training for learning keyboard short-cuts.  Including Vim & Emacs
  • Github Pre-commit hooks StackOverflow topic discussing setting up and testing pre-commit hooks on Githubs JSON API.  Github actually has a pretty decent into into some of their other hooks as well, see Post-Receive Hooks and Testing webhooks
  • Complex Responsive webapps more of a personnel anecdote than a tutorial but has some really good information on building responsive websites… after the fact.
  • Introducing Foreman Start-up manager for multi executable webapps.  Specifically in Ruby

Debian Linux

  • mini-dinstall On-line man page for mini-dinstall.
  • dh_install StackOverflow explanation of setting up a simple direct copy install rule for deb packages.  Particularly useful for web deployment packages.
  • gem2deb Github project page for gem2deb software.  Helps in created deb packages from Ruby gems also check out the Debian Ruby Packaging Team Wiki.
  • deb package building Debian.net forum post covering package building.  Some useful tips from here.  Honestly, RPMs are still my preferred method for building software packages.
  • debchange manpage debchange is probably the simplest way to create changelog entries in Debian.  Changelog formatting (a requirement to build packages) is a seriously painful process without this.
  • Debian Maintenance Guide This is chapter 4 that specifically covers debian directory file requirements when building deb packages.  Chapter 6 covers building, the Mentors FAQ  and Package FAQ have some good information as well.
  • Debian Admin Handbook Particularly this chapter (15.3) covers setting up an APT package repository using mini-dinstall.
  • Using dput with mini-dinstall A quick tutorial on using these two systems together.
  • dpkg cheatsheet Because I didn’t know how to do rpm -qi and rpm -qa in dpkg.
  • Debian Ruby Packaging Team Info includes tools, tips, standards, and links to information about packaging Ruby gems on Debian.

A Date Which Will Live In Infamy

Date string conversion is fairly painless in JavaScript but sometimes the sheer number of options can be a little annoying to remember.  Below is a table of date display/conversion functions generated from

new Date(“2013-02-19T21:03:39.818Z”).

Hopefully this is helpful to someone else who doesn’t want to look up the output of each of these options.  One more note, these are outputs for US locals in the Central time-zone; other locals and other time-zones would very accordingly.

Date Function Output
 toString() Tue Feb 19 2013 15:03:39 GMT-0600 (CST)
toDateString()  Tue Feb 19 2013
toGMTString() Tue, 19 Feb 2013 21:03:39 GMT
toISOString() 2013-02-19T21:03:39.818Z
 toUTCString()  Tue, 19 Feb 2013 21:03:39 GMT
toTimeString() 15:03:39 GMT-0600 (CST)
 toLocaleString() Tue 19 Feb 2013 03:03:39 PM CST
 toLocaleDateString()  02/19/2013
 toLocaleTimeString()  03:03:39 PM
 toJSON()  2013-02-19T21:03:39.818Z
 valueOf()  1361307819818
toSource() (new Date(1361307819818))

Of Liberty and Theater

The first and second Amendments are really the counter weights to Democracy. Those who ignore the second will ultimately loose the first, and a society that limits the first proves itself unworthy of the second. My biggest fear, with regard to the tragedies of late, is that in an attempt to save “just one child” we will surrender their future freedoms for security theater. That liberty will be forfeit at the alter of good intentions, doing nothing more than to make a world that is little less worthy of their sacrifice.