<Just-Words>

I write about tech, hacking, programming and stuff :)

Unicorn init scripts with RVM »

On our production servers we use the Unicorn application server together with RVM and nginx to serve our Rails sites. This means that we need an init script for each application in case of a server restart.

Since we’re using RVM we have each unicorn process run under an unprivileged user…

httprint.awk

 While investigating a burst of HTTP 500 errors for static files, I created a small script that will visualize the http status codes on a timeline. 
awk '/(18|19|2.)\/Sep/ { 	# First pattern I used was /somefile.css/ and then just /.css/ and then matching dates 
  ts=substr($4,2,14); 		# 14 = hour ; 16 = 10 minutes ; 17 = minutes
				# $4 = "[22/Sep/2014:09:01:49"
				#        2            H TM
				# H = substr($4,2,14) = 22/Sep/2014:09
				# T = substr($4,2,16) = 22/Sep/2014:09:0
				# M = substr($4,2,17) = 22/Sep/2014:09:01
  if (ts != prevts) {
    printf("\n%s ", ts);	# Print timestamp ever hour/10 minutes/minute depending on above substr length
    prevts=ts;
  };
  if ($9 == 200) {                  printf("."); } # OK
  else if ($9 == 500) {             printf("!"); } # Error
  else if ($9 == 404) {             printf("?"); } # Not found
  else if (substr($9,1,1) == "3") { printf(">"); } # Redirect
  else {                            printf(substr($9, 1,1)); } # Unknown, print first number of status code
}' ACCESS_LOG_FILE_HERE | sort -k1 ; echo          # sort -k1 because if you do awk '...' access_log* then they come in wrong order

Example for minute granularity:
22/Sep/2014:14:06 !.!!..
22/Sep/2014:14:08 >.
22/Sep/2014:14:09 .................
22/Sep/2014:14:10 ..............
22/Sep/2014:14:11 ............
22/Sep/2014:14:26 >............
22/Sep/2014:14:27 ............
22/Sep/2014:16:17 .
22/Sep/2014:16:18 ...........?.....
22/Sep/2014:16:20 ?>.....>?.....>>?....>.
22/Sep/2014:16:21 >?....>.>?...>..?...>.?...>..?...>..?.....>>

Vs hour granularity

22/Sep/2014:11 ......!!!!!!!!!!!!
22/Sep/2014:12 >........>...........!!!!!!................................
22/Sep/2014:13 ........................................................................>>
22/Sep/2014:14 !!.!!..>............................................>........................
22/Sep/2014:16 ............?.....?>.....>?.....>>?....>.>?....>.>?...>..?...>.?...>..?...>..?.....>>>......................
It helped me figure out when the problems started and wanted to share if this helps someone sometime in the future. You can use it to monitor http requests in real time:

/var/log/httpd $ tail -n 500 -f SOME_LOG_HERE | awk ‘/(19|2.)\/Sep/ { ts=substr($4,2,17); if (ts != prevts) { printf(“\n%s “, ts); prevts=ts; }; if ($9 == 200) { printf(“.”); } else if ($9 == 500) { printf(“!”); } else if ($9 == 404) { printf(“?”); } else if (substr($9,1,1) == “3”) { printf(“>”); } else { printf(substr($9, 1,1)); } }’

Little Snitch, round two »

httpshaming:

(We already posted about Little Snitch phoning home on port 80, but this is even worse. C’mon, Objective Development team, we love Little Snitch and want it to be secure!)

“I was interested in what app was used to show the shameful HTTP-only traffic and found in one of the old posts that it was