All Your Links Are Belong to Us

I have over 200 tabs open between three computers… the insanity must stop. Link dump to follow:

Software Engineering

  • Road to Continuous Delivery – Great article covering the different stages development shops go though to get to continuous delivery.  Provides a great starting checklist of what to work on while improving your software delivery process.
  • Overview of Micro Services – Micro services have become massively popular with the advent of Node.js.  Intro to the concepts are reasoning for using micro services.
  • Continuous Code Coverage with GCC, Hudson, and Googletest- Part of continuous delivery is continuous testing of your code base.
  • Trashing ChromeOS – Guide to building ATOM processor build-chain testing servers from Chromebooks.
  • Don’t Be Afraid of Functional Programming - The parts of JavaScript I actually like are its functional programming capabilities (callbacks and first class functions baby.)  OOP programmers get a little scared of things like Lisp, but they shouldn’t be.
  • Tessel – A JavaScript compatible (via Node.js) microcontroller.  Wow.. just wow.. Includes wifi built-in and shield/breakout board compatability through Node modules.

Source Code

  • DevDocs – Seriously hell yeah!  Online, simple, clean, extensive, software documentation for programming languages and libraries.  Seriously, browser pin this now!
  • Sourcegraph – Search tens of thousands of code examples.
  • Explainshell – Type some bash in, it explains what it is and what it does.  Think man pages for the internet age.
  • LibCurl API Reference – Using Curls powerful web functionality inside of C.

Vim

  • Smart Tabs - Tabs in the leading spaces, spaces for everything else.  The way GOD intended code to be structured!
  • 76 Vim Shortcuts – Some of these I already knew, but like all things Vim… there is always more to learn.
  • Mapping Standard Shortcuts – Things like Ctrl + c for copy.  I don’t actually remap my Vim shortcuts to match but I like the article because it explains what those shortcuts are used for by default.
  • Cheat Sheet – My current favorite Vim cheat sheet.  Simple and easy to search for things… and NO ads.
  • Vim-Adventures – Learn Vim shortcuts while playing an online game.
  • Exuberant CTags – You need more jumping around in your code.  Make it easy to swtich between headers, declarations, and usage locations in your projects.

C Programming

  • JANSSON – JSON in C.  Seriously freaking awesome!
  • Ncurses Programming in C - Another Linux Documentation Project about programming.  Ncurses is a command line interface for gcc.
  • Coding Unit, C Tutorial - Great introduction to C programming.  Simple examples and be sure to check out the comments below each section.
  • TutorialsPoint, The C – Another introductory guide to C development.  Better as a reference guide that the one from Coding Unit.
  • TDD in C – Love test driven development (not so much behavior driven development.) Simply way to due it in C without external dependencies.  Here is a list of tools if you do want to use external libraries for TDD.
  • Beginners Guide to Linkers – When compiling doesn’t kill you, the linker will.
  • Going from C to Go – I really like Go, but am currently doing a fairly serious C project.  Just in case I ever want to port it.

Terminal/Serial Programming

  • Terminos – C serial interface library for GCC.  The link is to the manpage with function examples.
  • Serial Programming – The Linux Documentation Project examples for serial programming in C.  Also look at the debugging section.
  • QTSerialPort – The Qt Library for Serial port communications.  Qt is easily the best C++ code library in existence.  While most people think of Qt only when coding GUI applications; its libraries are extensive enough to use for ANY application… even from the command line.
  • Stackoverflow Serial Examples in C – Couple good examples and they got me some of my first working C Serial code.
  • RS-232 Library - Works on both Linux and Windows.
  • Serial Programming in Linux – Wikibooks book style tutorial on serial programming in Linux.  Here is the specific section on Terminos.
  • WiringPi – Serial programming interface on the RaspberryPi.  Very nice if you are using the Pi breakout pins.
  • Serial Example – Another quick example by tty1.

Cloud/Web

  • aws – The Amazon Web Service command line tool.  One of the better reference pages I have seen on it.
  • Web Graphs & Visualization – 30 tools for web based data visualization. Both Sas and Open Source tools listed.
  • Let’s Make a Bubble Map – Thematic mapping.  Includes a link to a D3 tutorial for creating bubble map.s
  • Camlistore – A personnel, decentralized, non-heiarchy based storage system that can be synced between cloud, phone, computer, and anything you can think of.
  • JSON Form Editor – Similar to a project I did myself a while back.  Automatically create forms based on simple JSON structures.  Makes it easy to to AJAX requests to build, populate, and check forms.

Cyber Security

SSH

  • Rsync & SSH - Combining two of the most powerful software utilities in existence to make backups.
  • StormSSH – I actually created a series of bash scripts to do SSH bookmarking.  Storm improves on this idea by directly editing your ssh config file with the stored entries. Wish there wasn’t a Python dependency.
  • SSH Kung Fu – Great tutorial covering some of the many of the one-off capabilities of SSH like remote folder mounting, port forwarding, and connection sharing.
  • Simplify With SSH Config – Good overview of some SSH configuration and setup options.  Plays well along side the SSH Kung Fu link above.
  • Autocomplete Hostnames – Process your hosts file under /etc/ as well as your config files for autocomplete.  I used my hosts file as a blacklist so this doesn’t work as well for me but the information was useful for the SSH bookmark system I made.
  • Commandline Fu SSH Autocomplete – A few ways to populate the autocomplete functionality for SSH.

Bash

  • Better Bash – Simple suggestions on making better bash scripts.
  • Command Tips & Tricks — Nice overview of some tip using Bash, Vim, networking, and general command line productivity.
  • Tmux Cheatsheet – If you know what tmux or screen are… then this is pretty helpful.  Otherwise you need to find out what tmux is.
  • Defensive Bash Programming – More and better ways to organize your bash programs.
  • BashGuide – Better bash than the Gnu Bash guide… or at least some people say so.  Evidently it has fewer “bugs” in its code examples.

Linux Bluetooth

  • Bluetooth on Fedora – A number of Bluetooth devices need re-pairing (or at least a user to be logged into the system) before they will connect after restart.  This is particularly frustrating for Bluetooth mice and keyboards.  Following the advice by the selected answer (as root0 solved the problem for me.  This one was annoying and hard to find out.
  • ThinkPad Compact Bluetooth Keyboard – Cannot wait until this keyboard driver is put in the mainline kernel so I don’t have to build it every time.  This keyboard allows me to use the same keyboard when using my Carbon X1 or when at my desk.

Misc

  • Clark DuVall – I don’t know who the guy it, but he has the most freaking amazing website I have ever seen!  Command-line junkie heaven.
  • Bluetooth Deadbolt – Something to keep my from having to carry another key?  Plus it lights up!
  • LibreOffice & Google Docs – Open, edit, and save/upload Google Docs directly in Open/Libre Office.  Also has support for WebDAV.
  • Switching from Photoshop to Gimp -  List of modifications to make the Gimp feel more natural for Photoshop users.
  • Industrial Strength Bubbles – Make person sized bubbles that last for several minutes.  I know what the kids and I are doing this weekend.
  • Open Energy Monitoring – Open source automation and energy monitoring.
  • Free your Android – As in Freedom, not beer.  Useful list of Free Software version of popular software on Android.
  • Shortcut Foo – Tutorials for quickly learning misc programming environment shortcuts.

Life is what you make of it

The most difficult aspect of software development for new programmers often has nothing to do with algorithm complexity or syntactical quarks; it’s all the other “stuff” associated with building, managing, and testing software systems.  When a developer steps into a existing business that already has a software stack to support the problem can be mitigated by relying on the institutional knowledge that the existing developers have formed over the course of maintaining their software.  I haven’t, in most cases,  had that fortune  in my career because either a) I was the companies first software engineer, or b) the existing software engineers had become proficient at a “alternative” software stack (and honestly, alternative is the kindest word I could come up with for Mainframe/Cobol.)

The longer a programming language has been around the more complex these build & management tools get.  The reasons are pretty simple.  The longer a language has been used, the more complex and more broad the uses of that language become.  Build tools generally start off pretty simple (make was originally an 8 line bash script for gods sake) but they must expand to cover more and more complex setups with more and more non-standard configurations.  In the most extreme cases the support tools even need to consider multiple platforms on multiple hardware configurations.  This problem can be exacerbated when a language needs to be “compiled” (and I use the term loosely) or works on “core” systems, meaning closer to the hardware, network, or data layer.*

C suffers from all of the above listed problems and more.  Having been around for around 40 years, in constant usage, on every platform ever made (super-computer to toasters), used for hardware drivers, operating systems, core network stacks, and even to create other programming languages; means that C can be the most complicated system ever supported by mankind.  I’m really not kidding about this.  More than one person has pointed out that the Linux kernel (95% pure C code) is many orders of magnitude more complex than sending a man to the moon is or even sending a woman to Mars will be.  Anyone who has had to create a Gnu build-chain supported C program from scratch has had to kill themselves learning the intricacies of make, automake, config, autoconfig, m4, autoreconfig, cmake, libtools, and autoheader.  Seriously, a “correct” Gnu C project with 1 header file and 1 c file has 26 buildchain files supporting it on initial setup.

Recently I have been doing some really interesting C development on micro-mobile platforms.  The first language I did significant (i.e. not a GWBasic MadLibs game) development on was C.***  My college experience with C was relegated to a couple hundred lines and using the up arrow to re-compile the program after changes.  Now my annoyance with with the autoconf build tools (and its many many gotchas) is replaced with the need to support cross-compiling, manage external libraries, and automate build deployments.  I have had to learn each of these tools and what it is they accomplish for me so I don’t have to re-invent the wheel.  Here are some of the more useful sources of information I have come across:

  • Gnu Autoconfig, Automake, and Libtools – by Gary V. Vaughan, Ben Elliston, Tom Tromey and Ian Lance Taylor.  Available as a Web Book it covers the entire build chain and practical usage of each of the tools.  Also does a great job of showing how modern technological development owes a huge debt to the flexibility and power these tools gave (and continue to give) C developers.
  • Gnu.org amhello – A “Hello World” tutorial for getting Autotools setup and configured in a simple project.  Great example for getting a full build setup running for C.  The full code of which can be found in the automake doc folder on Linux systems, generally something like /usr/share/doc/automake/amhello-x.x.x.tar.gz.
  • Clemson Automake by Example – Old article (the pages images are all broken) that walks through a simple C program and its build chain.  Excellent tutorial for getting a notice programmer setup with a distributable and effective build environment.
  • Autotools Mythbusters – Practical, if high level, overview of autotools and its associated components.  There is an Appendix with a list of examples that is particularly outstanding.  Think stackoverflow for autotools that has been aggregated into a Cookbook.
  • Simple Makefile Tutorial - A newbie guide to creating Makefiles for building software.  If I include code examples in the project documentation I will generally create a simple Makefile that will build the examples with a “make someexample”.
  • Martin Mann’s HowTo Autotools – The examples are in C++ but the step by step process to add functionality to the autotools build chain is outstanding.  Especially useful if you have figured out some of the basics already.

Finally, because setting up and creating the necessary files for getting a C project started in libtools/automake are so annoying, I decided to create a single file bash script to do the work for me.  You can find it as a gist on github.  You can download and run it by doing a:

wget http://tinyurl.com/brockers-cmaker -O ~/bin/cmaker && chmod +x ~/bin/cmaker

On the Linux command line.  Then create your new C/Autoconf project with cmaker init newprojectname.  My primary concern with the script was that is should need NO outside dependencies besides libtools/automake itself and that it has everything needed to start the autoreconf –install, ./configure, make process.  Hopefully I will add some additional functionality to it soon.

* As an example, look at Perl.  It initially started as a glue language to allow developers to piece together software solutions in a single language instead of having to create divergent sed, awk, and grep scripts in sh**.  Then the WWW took off and the little glue language became the core component of the most powerful websites on the planet.  Perl went from being a support language to the core language of all things http.  The number of tools exploded.  CGI.pm, mod_perl, and DBI gave the developer massive power but managing these libraries in production created a boom of support tools (kids these days forget that cpan was the ruby gems/bundler of its day.)

** As a side note that last sentence sounds more like a caveman grunting then a discussion of software development tools.

*** OK, ttechnically it was C++ but our CS chair was a former NASA Chief Engineer who basically taught us C using a C++ compiler… with a little Class thrown in.  I think my first object was linkedList with methods push and pop.

That’s not writing at all

Cheatsheet of some of the more useful vim commands I have run into lately:

  • ci(, ci{, ci<, ci", ci' – Delete and insert into text between (, {, <, “, etc.. brackets.  Really useful for things like function arguments and quoted text.
  • ca(, ca{, ca<, etc... - Same as above but removed the "bracket" as well.
  • yi(, di{, va<, .. - Yank content between ( and ), delete content betwen { and }, select content between < and > including brackets, etc..
  • ZZ – Quick same/close, works the same as :x.
  • * – Search for word under the cursor.
  • gv – Re-select previous selection block
  • q<letter> @<same letter> – Record a macro with q (named letter) and play it back with @.

Always Remember…

The Bill of Rights is a literal and absolute document. The First
Amendment doesn't say you have a right to speak out unless the
government has a 'compelling interest' in censoring the Internet. The
Second Amendment doesn't say you have the right to keep and bear arms
until some madman plants a bomb. The Fourth Amendment doesn't say you
have the right to be secure from search and seizure unless some FBI
agent thinks you fit the profile of a terrorist. The government has no
right to interfere with any of these freedoms under any circumstances.
--Harry Browne

Virtual Private Networking in AWS

Been doing lots of VPN setup and configuration lately, especially inside of Amazon Web Service (AWS) Virtual Private Clouds (VPCs.)  They have a built-in VPN capability using IPSec but it generally seems specifically focused on device-to-device (D2D) configurations.  Depending on the need I have turned up StrongSwan and/or OpenVPN as a solution.

OpenVPN has an advantage of being able to do SSL VPN on 443 making it look exactly like HTTPS web traffic (effectively making it unbreakable by network administrators.)  Things like proxy-servers don’t even know you are creating a VPN tunnel.  However, on Windows OpenVPN client software has to be installed to use it.

StrongSwan is a IPSec VPN option that works well with existing P2P VPN systems.  The native Windows VPN tools work out of the box with a standard StrongSwan configuration (as long as your certs have been signed by a trusted CA.)  Performance is also very good.

So far, I really really like OpenVPN as once it is configured it works everywhere, regardless of network policy or ISP limitations.  Linux Network Manager has built in support for it making is very very easy to configure clients to use it as well.  That said, for IPSec configurations needing to connect to Windows Clients; StrongSwan has been my go-to solution.

Useful links follow:

Linux StrongSwan Server

Workstation StrongSwan Setup/Install Client

OpenVPN on Ubuntu

drift toward unparalleled catastrophe

My home configuration has two Planar 20″ monitors as my primary display.  They have worked fairly well with the exception that any sudden change in input signal seems to cause them to freak out and changed their sync levels to non-standard ranges.  Resetting them is the fix but Planar is kind enough to NOT mention how to do that in any of their documentation.  So, for the benifit of mankind here is the process for resetting a Planar PL2010MW to factory default>

Unplug the monitor.  Counting from the left, there are five buttons on the bottom of the monitor (the right most being the power button.)  Press the second and fourth buttons from the left and hold them down while plugging the monitor back in.  Count to five, and release.

Other models of Planar use the second and third buttons with variations of releasing immediately after plugging in; or waiting until the main power light turns green.  In addition, if you are using some versions of Linux you may have to restart X before you see the minor in your hardware setup.

All I see are tabs

Cloud:

  • BitTorrent Sync – Multiple source file syncing using bit torrent client.  Thank of it as headless Dropbox.
  • Own Cloud – Open Source personnel cloud solution. Includes things like data, music, contacts, calendar and can even be used by multiple clients. Even set it up on your own server.
  • SparkleShare – Dropbox like functionality on Linux, Mac, and Windows systems. Includes versioning as well.
  • Gmail Forwarder -  Correctly configure gmail when using your own smtp settings, domain, and email forwarder.
  • Github For Everything – Using Github to manage everything in your company; from your hiring process to your internal documentation.
  • Using Gnu StowStow is a open source tool for managing your dotfiles in a universal way across multiple machines.  This also simplifies the process of using a version control system to track your dotfile history.
  • Git-Annex – Headless, versions, unlimited, decentralized file syncronization for Unix systems.  Based on Git and includes a mobile app.  Possibly the best replacement for Dropbox available anywhere.

App Dev

  • Apple App Distribution – All 100 freggin pages of it.  Includes beta testing and is almost like developing software back in the 90′s.
  • Android App Distribution – Eight pages and you can even using Google Groups to manage your beta test groups.  Includes automatic updates.
  • Ruby Rack nginx – Very clean, very simple example of setting up a Ruby Rack nginx configuration.
  • Source Code Comments – A list of the most humorous source code comments people have read.
  • Testing Code, Simply – I love this post.  Best simplification of how/why TDD should be used.  The examples can even be modified to allow testing of things like Bash or VBA.

Bash & SSH

  • More cool bash commands and shortcuts.
  • Need a web server, how about a single line of bash.
    while true; do { echo -e 'HTTP/1.1 200 OK\r\n'; cat index.html; } | nc -l 8080; done
  • Resetting an unresponsive SSH session.
    newline ~.
  • Setting up SSH to use shared concurrent connections.  Came from a tutorial on speeding up Git, but useful everywhere.
  • More SSH Awesomeness – This was how I learned ssh-copy-id.  Lots of other amazing advice.
  • Improve SSH Key Security – Things to do AFTER you have placed a passphrase on your SSH keys.  You have done, at least, that?  Right??
  • Passing SSH commands in git clone – Stackoverflow reponse on how to configure .ssh/config options for specific hosts.  They, in tern, get picked up by git.

Go

  • Go Tutorial Exercises – I have really enjoyed GO lately.  C language power with a language actually built for multi-core processors in a network connected world.
  • Effective Go FAQ – Some really great tutorials and information for developers trying out a new language.
  • Hermans Go – Project Euler solutions written in Go.  Great example code for learning algorithms in Go.

Vim

  • Nouns & Version – Understanding the basic structure of how VIM works.
  • Colons are Bad – How to stop using colon commands in VIM.
  • YADR – Sample dot files, vimrc, git support and other useful vim tools.
  • Vim Bookmarks – How to use and manage bookmarks in Vim.
  • Yankstack – A plugin to give kill ring capabilities to the Vim.
  • Block Shift – Visual block shifting in Vim.  Tab and un-tabbing, spacing, and block selections are all covered.
  • Vim Adventures – Learn Vim while playing a video game.
  • Awesome Vimrc – That is the name its developer gave it, not mine.  Still, it is a pretty cool, VERY clean vimrc file that has a lot of good examples in it.  Also can be found on github.

Misc

  • Programming Books – List of freely available programming books.
  • Effing Package Manager – Create rpm, deb package building directly from gems and bundler.
  • View your Axciom Data – Axciom is one of the largest data brokers of personnel information on the planet.  This website allows you to see the data that Axciom has on you.  The downside?  They get to keep the data you have to submit to see the data they already have.
  • Large Distributed System - Advice from people who build Google.
  • Faynmen Lectures on Physics – Everything you have ever wanted to know about almost everything that we think we know.
  • Nginx Secure Configuration – Setting up and securing nginx with ssl.
  • Debugging Broken postinst on Debian – Basically the postinst file gets installed anyway, so you just need to edit it on the semi-installed machine and then run it again… until it is fixed.
  • Bruce Schneier’s Sept 2013 Cryptogram -  Read this if you want a better explanation of why you should be VERY VERY afraid of what the NSA and large internet companies are doing.  Some articles are very technical but others are surprisingly approachable to the lay person.

This is how the world end

I have a family member who recently said to me that if I posted pictures of them on Facebook, they would stop speaking to me.  This, entirely understandable, concern stems from their conscious concern that personnel information collection by large companies has a tendency to be abused.  Once you have surrender your privacy it is nearly impossible to get back.

What made the conversation stand out to me wasn’t their “fear” of business; but that this particular family member is one that inherently trusts government to solve this (and many other) issues.  There seems to be a fundamental disconnect between the perceived danger from business and the real danger of government.

Coca-cola cannot force my soda consumption (or limit the size of my cup.)  Google cannot regulate which sites I am allowed to visit, or what the content of those sites can be.  Phillip-Morris is entirely unable to limit the extent of my free speech by defining who is, or is not, a “legitimate” reporter.  And while Facebook may want to use your personnel information to sell you crap, or profile you activities; it doesn’t have the ability levy punitive damages, listen in on any phone conversation you have ever had, or target you with a drone strike.

One’s personnel privacy should certainly be protracted, but a healthy fear of the abuse of capitalism should always be tempered with a real fear of the only institution that has the ability to use force against us.  An institution that has demonstrated time and time again that it abuses that force to the detriment of both our privacy and our liberty.

I am not a link bot… I hope

Once again my desktop has become to cluttered with links.  Here are some of the ones I have been using the last couple weeks.

Vim

  • Vim Cheet Sheet – A short list of useful Vim commands & short-cuts.
  • Vim copy and past commands – Setting blocks, yank, paste, cut, etc.. in vim
  • Vim word completion  – Found this more useful after binding it the completion command to the tab key (aka bash mode.)
  • Remove unwanted spaces – Because some “people” think using spaces instead of tabs is a good idea.
  • Accessing the System clipboard in Vim – Because Vim registers do not necessarily map to the OS clipboard.  The quick summary is that I would strongly recommend putting the following alias in your .bashrc if type “gvim” > /dev/null; then alias vim=”gvim -v”; fi then make sure you have gvim installed.
  • Using Vim Registers – Actually using the registered mentioned above.
  • Pasting in Visual mode – Using registers is great but not really useful if you keep having to switch back to command mode to use them.

DBus

Ruby

  • Singing with Sinatra Pt. 2 – Sinatra is a ultra simplified application server environment for Ruby.   Think Rails only about 1/10th its size.  This was the best of the tutorials I found for it.
  • Thin Server Production and static files - This little blurb was something I caught on StackOverflow and knew I would need for later as our production system is running into the same issue.
  • fpm (freggin package manager) – Tool for creating deb/rpm packages from lists of filesystem files.  Particularly useful for gem files (it even has it as an option.) I am in the process of moving over my existing ruby build scripts over to fpm.

Debian

  • Creating Meta Packages - Meta packages are simply empty deb packages that contain nothing but a list of dependancies.  This way you can create a batch of files to be installed for a given purpose (like installing KDE Desktop.)
  • equivs-control man page- Used in the creation of Meta packages
  • Binary Package building tutorial for Debian – The deb build package environment basically builds itself around have source for all software.  This is a problem for packaging non-open source programs that don’t provide a source.  This is a tutorial for how to do it.
  • Template Changes file – Debian apt repositories generally work with .changes files to actually publish their packages.  This is an example of a changes file for the package dpkg-ruby.
  • Create you own apt repository – Includes information on upload support (which uses changes files mentioned above.)
  • Creating a basic Ruby application structure – How to create you base dependencies, directory structure, and file-system layout for a base Ruby project.