Tuesday, September 6, 2016

Monitoring Raspberry Pi using my DigitalOcean VPS

So, this is a simple one and just felt like sharing in case someone wants to save time writing those one-liners.

I had a faulty raspberry pi power supply and my Rpi used to hang unexpectedly. Here is what I did to monitor uptime of my Rpi.


  • Host a simple text file via nginx on my DigitalOcean VPS.
➜ $?=0 /home/shadyabhi [ 1:20PM] % cat /var/www/html/rpihealth
GOOD

>>>  0s elasped...
➜ $?=0 /home/shadyabhi [ 1:20PM] %
  • Write a simple systemd service in Rpi to curl this file. 
➜ $?=0 /home/shadyabhi [ 5:21PM] % sudo cat /usr/lib/systemd/system/send_latest_ip.service
[Unit]
Description=Latest IP sender

[Service]
User=shadyabhi
ExecStart=/usr/local/bin/send_uptime.sh

[Install]
WantedBy=multi-user.target

>>>  0s elasped...
➜ $?=0 /home/shadyabhi [ 5:21PM] % cat /usr/local/bin/send_uptime.sh
#!/bin/bash

while [ 1 ]; do
        curl https://abhijeetr.com/rpihealth
        sleep 1
done

>>>  0s elasped...
➜ $?=0 /home/shadyabhi [ 5:21PM] % sudo systemctl enable send_uptime
  • A sample logline for a health checks looks like this.
106.51.128.57 - - [06/Sep/2016:13:37:46 -0400] "GET /rpihealth HTTP/1.1" 200 5 "-" "curl/7.50.0" "-"
  • A cronjob on my VPS to send me notifications via pushover if there are no log lines for this health check in the last 10 seconds. (using https://github.com/jnwatts/pushover.sh)
* * * * * sudo /usr/bin/perl -MDate::Parse -ne 'print if/^.*- - \[(.*?)\] .*?/&&str2time($1)>time-10' /var/log/nginx/access.log | if ! /bin/grep -q "/rpihealth"; then /home/shadyabhi/pushover.sh/pushover.sh -t "RPi Health" "Rpi is down"; fi

Now, whenever my Rpi is down, I get push notifications on my phone.





Saturday, February 27, 2016

Golang: Creating HTTPS connection via proxy

I'm writing this post because I had to read a bit of golang code to figure out how to go about doing this.

User familiar with crypto/tls/ will notice that there's already a function available to establish a TLS connection, tls.Dial. However, it doesn't have any option to specify proxy. Why? Because, TLS connection has nothing to do with proxy, TLS is available as a addon to an already existing TCP connection. It exists one level below HTTP, the application layer protocol. That's exactly the reason tls.Dial function is not present in the net/http package too.

A simple way to do a request is http.Client.Get. (). Let's see what it does.
func (c *Client) Get(url string) (resp *Response, err error) {
    req, err := NewRequest("GET", url, nil)
    if err != nil {
        return nil, err
    }
    return c.doFollowingRedirects(req, shouldRedirectGet)
}

It simply does a request using NewRequest function. All it does is returns a proper Request struct depending on various things like type of request, URL to open, headers etc etc. Depending on various StatusCode, headers etc received, request is handled and ultimately (*Client).send is called. This function handles cookies and then actually calls the send function in the net/http package which is not available to the outside world.

func (c *Client) send(req *Request, deadline time.Time) (*Response, error) {
 if c.Jar != nil {
  for _, cookie := range c.Jar.Cookies(req.URL) {
   req.AddCookie(cookie)
  }
 }
 resp, err := send(req, c.transport(), deadline)
 if err != nil {
  return nil, err
 }
 if c.Jar != nil {
  if rc := resp.Cookies(); len(rc) > 0 {
   c.Jar.SetCookies(req.URL, rc)
  }
 }
 return resp, err
}

Interesting thing to note here is the c.transport() that we're passing. This function returns the transport being used for the current client.
func (c *Client) transport() RoundTripper {
 if c.Transport != nil {
  return c.Transport
 }
 return DefaultTransport
}
From the docs:-
Transport is an implementation of RoundTripper that supports HTTP, HTTPS, and HTTP proxies (for either HTTP or HTTPS with CONNECT).
So, this is where we can set proxies. RoundTripper is an interface and pointer to http.Transport implements that interface.

RoundTripper is an interface representing the ability to execute a single HTTP transaction, obtaining the Response for a given Request. The http module providers an implementation of RoundTripper, http.Transport
type Transport struct {

    // Proxy specifies a function to return a proxy for a given
    // Request. If the function returns a non-nil error, the
    // request is aborted with the provided error.
    // If Proxy is nil or returns a nil *URL, no proxy is used.
    Proxy func(*Request) (*url.URL, error)
We've found our Proxy. SUCCESS!

So, all we need to do is, create a custom http.Transport and then a http.Client that uses this newly created http.Transport.

Now onto the real task at hand.
  • Create a custom transport using proxy
  •         // Create proxy
            proxyURL, _ := url.Parse(*proxy)
    
            transport := http.Transport{
                Proxy:           http.ProxyURL(proxyURL),
                TLSClientConfig: &tls.Config{},
            }
    
  • Create the http.Client object
  •         client = http.Client{
                Transport: &transport,
            }
    
  • Call http.Client.Get
  • resp, _ := client.Get("https://www.google.co.in")
    
Using what we have learnt, here's the code below that connects to a live HTTPS endpoint, fetches its certificate and shows the days to expiry for that cert. 
package main

import (
        "crypto/tls"
        "flag"
        "fmt"
        "net/http"
        "net/url"
        "time"
)

const timeout time.Duration = 10

func main() {
        // Parse cmdline arguments using flag package
        server := flag.String("server", "abhijeetr.com", "Server to ping")
        port := flag.Uint("port", 443, "Port that has TLS")
        proxy := flag.String("proxyURL", "", "Proxy to use for TLS connection")
        flag.Parse()

        // Prepare the client
        var client http.Client
        if *proxy != "" {
                proxyURL, err := url.Parse(*proxy)
                if err != nil {
                        panic("Error parsing proxy URL")
                }
                transport := http.Transport{
                        Proxy:           http.ProxyURL(proxyURL),
                        TLSClientConfig: &tls.Config{},
                }
                client = http.Client{
                        Transport: &transport,
                        Timeout:   time.Duration(time.Millisecond * timeout),
                }

        } else {
                client = http.Client{}
        }
        // Now we've proper client, with or without proxy

        resp, err := client.Get(fmt.Sprintf("https://%v:%v", *server, *port))
        if err != nil {
                panic("failed to connect: " + err.Error())
        }

        fmt.Printf("Time to expiry for the certificate: %v\n", resp.TLS.PeerCertificates[0].NotAfter.Sub(time.Now()))
}


 Cheers !!

Friday, January 15, 2016

Code Snippet: A redirection service in Go

Recently, I've been very fascinated with Go. That language is so easy to write and as it's statically compiled, less changes of bugs too. So, I keep finding excuses to write Go code.

I needed a service to redirect clients which runs inside a docker container. One obvious option is to run apache inside docker with proper configs but as I enjoy writing Go code, I wrote this really simple program that acts like a redirection service. Later, I just copy this compiled binary into docker container.
  • Specify location and port from command-line to set the location to redirect to. 
  • Each client is handled in it's own goroutine, that means, it's scalable. 
  • That few lines of code are so elegant. I love Go.

Code:- ( https://github.com/shadyabhi/redirection_service )

package main

import (
	"flag"
	"log"
	"net/http"
	"strconv"

	"github.com/asaskevich/govalidator"
)

// redirect function does the redirection
func redirect(w http.ResponseWriter, r *http.Request, location string) {
	w.Header().Set("Location", location)
	w.WriteHeader(http.StatusFound)
}

func main() {
	// Handle cmdline arguments
	location := flag.String("location", "localhost:80", "The URI at which the ")
	port := flag.Int("port", 80, "Port at which the http server binds to")
	flag.Parse()

	// Validate
	isURL := govalidator.IsURL(*location)
	if !isURL {
		log.Fatal("The location you provided is not a valid URL")
	}

	log.Printf("Starting web server... Location: %s, Port: %d", *location, *port)

	http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
		redirect(w, r, *location)
	})

	err := http.ListenAndServe(":"+strconv.Itoa(*port), nil)
	if err != nil {
		log.Fatal("Error starting web server. ", err)
	}
}

Usage:-

% ./redirection_service -h
Usage of ./redirection_service:
  -location string
        The URI at which the  (default "localhost:80")
  -port int
        Port at which the http server binds to (default 80)
% sudo ./redirection_service -location=https://abhijeetr.com
2016/01/15 09:33:54 Starting web server... Location: https://abhijeetr.com, Port: 80

Tuesday, January 12, 2016

Code Snippet: Fetch certificate information from a live endpoint in Go

I generally try to post snippets on my blog that I don't find on a Google search. This is like one of those snippets.

This sample code fetches the time to expiry for a certificate that's currently deployed at a live endpoint. 

Code:-

package main

import (
 "crypto/tls"
 "flag"
 "fmt"
 "time"
)

func main() {
 // Parse cmdline arguments using flag package
 server := flag.String("server", "abhijeetr.com", "Server to ping")
 port := flag.Uint("port", 443, "Port that has TLS")
 flag.Parse()

 conn, err := tls.Dial("tcp", fmt.Sprintf("%s:%d", *server, *port), &tls.Config{})
 if err != nil {
  panic("failed to connect: " + err.Error())
 }

 // Get the ConnectionState struct as that's the one which gives us x509.Certificate struct
 connectionState := conn.ConnectionState()

 fmt.Printf("Time to expiry for the certificate: %v\n", connectionState.PeerCertificates[0].NotAfter.Sub(time.Now()))
 conn.Close()
}

Usage:-

% ./ssl_certs -h
Usage of ./ssl_certs:
  -port uint
     Port that has TLS (default 443)
  -server string
     Server to ping (default "abhijeetr.com")
% ./ssl_certs
Time to expiry for the certificate: 1843h21m3.857177591s

Thursday, July 24, 2014

Accessing individual files from a "dd backup" of a encrypted hard disk/partition

Whenever I do something risky with my laptop like resizing partitions, I always do a full backup of the hard disk by running:
dd if=/dev/sda of=/path/to/backup/directory/image.img bs=4M
from a live archlinux USB pendrive.

Restoring from a backup like that is super easy.
dd if=/path/to/backup/directory/image.img of=/dev/sda bs=4M
Works perfectly. Every single time. Except licensing issues with Windows installation if you've a dual boot but WHO CARES!!!

In my case, one of my partitions /dev/sda5 is LUKS encrypted & if I want to access files from that partition using the .img file, things are not very straight forward but easy. For reference, this is what you'll typically do.
➜ 0 /home/shadyabhi [ 9:40PM] % sudo modprobe loop
➜ 0 /home/shadyabhi [ 9:40PM] % sudo losetup -v /dev/loop0 ./official_laptop_backup.img
➜ 0 /home/shadyabhi [ 9:40PM] % ls /dev/loop0*
/dev/loop0
➜ 0 /home/shadyabhi [ 9:40PM] % sudo partprobe /dev/loop0
➜ 0 /home/shadyabhi [ 9:40PM] % ls /dev/loop0*
/dev/loop0  /dev/loop0p1  /dev/loop0p2  /dev/loop0p5  /dev/loop0p6  /dev/loop0p7
➜ 0 /home/shadyabhi [ 9:40PM] % sudo mkdir /mnt/my_encrypted_partition
➜ 0 /home/shadyabhi [ 9:40PM] % sudo cryptsetup luksOpen /dev/loop0p5 encrypted_partition
Enter passphrase for /dev/loop0p5: 
➜ 0 /home/shadyabhi [ 9:40PM] % sudo mount /dev/mapper/encrypted_partition /mnt/my_encrypted_partition 
➜ 0 /home/shadyabhi [ 9:40PM] % sudo ls /mnt/my_encrypted_partition | wc -l
28
➜ 0 /home/shadyabhi [ 9:41PM] %