Saturday, February 27, 2016

Golang: Creating HTTPS connection via proxy

I'm writing this post because I had to read a bit of golang code to figure out how to go about doing this.

User familiar with crypto/tls/ will notice that there's already a function available to establish a TLS connection, tls.Dial. However, it doesn't have any option to specify proxy. Why? Because, TLS connection has nothing to do with proxy, TLS is available as a addon to an already existing TCP connection. It exists one level below HTTP, the application layer protocol. That's exactly the reason tls.Dial function is not present in the net/http package too.

A simple way to do a request is http.Client.Get. (). Let's see what it does.
func (c *Client) Get(url string) (resp *Response, err error) {
    req, err := NewRequest("GET", url, nil)
    if err != nil {
        return nil, err
    }
    return c.doFollowingRedirects(req, shouldRedirectGet)
}

It simply does a request using NewRequest function. All it does is returns a proper Request struct depending on various things like type of request, URL to open, headers etc etc. Depending on various StatusCode, headers etc received, request is handled and ultimately (*Client).send is called. This function handles cookies and then actually calls the send function in the net/http package which is not available to the outside world.

func (c *Client) send(req *Request, deadline time.Time) (*Response, error) {
 if c.Jar != nil {
  for _, cookie := range c.Jar.Cookies(req.URL) {
   req.AddCookie(cookie)
  }
 }
 resp, err := send(req, c.transport(), deadline)
 if err != nil {
  return nil, err
 }
 if c.Jar != nil {
  if rc := resp.Cookies(); len(rc) > 0 {
   c.Jar.SetCookies(req.URL, rc)
  }
 }
 return resp, err
}

Interesting thing to note here is the c.transport() that we're passing. This function returns the transport being used for the current client.
func (c *Client) transport() RoundTripper {
 if c.Transport != nil {
  return c.Transport
 }
 return DefaultTransport
}
From the docs:-
Transport is an implementation of RoundTripper that supports HTTP, HTTPS, and HTTP proxies (for either HTTP or HTTPS with CONNECT).
So, this is where we can set proxies. RoundTripper is an interface and pointer to http.Transport implements that interface.

RoundTripper is an interface representing the ability to execute a single HTTP transaction, obtaining the Response for a given Request. The http module providers an implementation of RoundTripper, http.Transport
type Transport struct {

    // Proxy specifies a function to return a proxy for a given
    // Request. If the function returns a non-nil error, the
    // request is aborted with the provided error.
    // If Proxy is nil or returns a nil *URL, no proxy is used.
    Proxy func(*Request) (*url.URL, error)
We've found our Proxy. SUCCESS!

So, all we need to do is, create a custom http.Transport and then a http.Client that uses this newly created http.Transport.

Now onto the real task at hand.
  • Create a custom transport using proxy
  •         // Create proxy
            proxyURL, _ := url.Parse(*proxy)
    
            transport := http.Transport{
                Proxy:           http.ProxyURL(proxyURL),
                TLSClientConfig: &tls.Config{},
            }
    
  • Create the http.Client object
  •         client = http.Client{
                Transport: &transport,
            }
    
  • Call http.Client.Get
  • resp, _ := client.Get("https://www.google.co.in")
    
Using what we have learnt, here's the code below that connects to a live HTTPS endpoint, fetches its certificate and shows the days to expiry for that cert. 
package main

import (
        "crypto/tls"
        "flag"
        "fmt"
        "net/http"
        "net/url"
        "time"
)

const timeout time.Duration = 10

func main() {
        // Parse cmdline arguments using flag package
        server := flag.String("server", "abhijeetr.com", "Server to ping")
        port := flag.Uint("port", 443, "Port that has TLS")
        proxy := flag.String("proxyURL", "", "Proxy to use for TLS connection")
        flag.Parse()

        // Prepare the client
        var client http.Client
        if *proxy != "" {
                proxyURL, err := url.Parse(*proxy)
                if err != nil {
                        panic("Error parsing proxy URL")
                }
                transport := http.Transport{
                        Proxy:           http.ProxyURL(proxyURL),
                        TLSClientConfig: &tls.Config{},
                }
                client = http.Client{
                        Transport: &transport,
                        Timeout:   time.Duration(time.Millisecond * timeout),
                }

        } else {
                client = http.Client{}
        }
        // Now we've proper client, with or without proxy

        resp, err := client.Get(fmt.Sprintf("https://%v:%v", *server, *port))
        if err != nil {
                panic("failed to connect: " + err.Error())
        }

        fmt.Printf("Time to expiry for the certificate: %v\n", resp.TLS.PeerCertificates[0].NotAfter.Sub(time.Now()))
}


 Cheers !!

Friday, January 15, 2016

Code Snippet: A redirection service in Go

Recently, I've been very fascinated with Go. That language is so easy to write and as it's statically compiled, less changes of bugs too. So, I keep finding excuses to write Go code.

I needed a service to redirect clients which runs inside a docker container. One obvious option is to run apache inside docker with proper configs but as I enjoy writing Go code, I wrote this really simple program that acts like a redirection service. Later, I just copy this compiled binary into docker container.
  • Specify location and port from command-line to set the location to redirect to. 
  • Each client is handled in it's own goroutine, that means, it's scalable. 
  • That few lines of code are so elegant. I love Go.

Code:- ( https://github.com/shadyabhi/redirection_service )

package main

import (
	"flag"
	"log"
	"net/http"
	"strconv"

	"github.com/asaskevich/govalidator"
)

// redirect function does the redirection
func redirect(w http.ResponseWriter, r *http.Request, location string) {
	w.Header().Set("Location", location)
	w.WriteHeader(http.StatusFound)
}

func main() {
	// Handle cmdline arguments
	location := flag.String("location", "localhost:80", "The URI at which the ")
	port := flag.Int("port", 80, "Port at which the http server binds to")
	flag.Parse()

	// Validate
	isURL := govalidator.IsURL(*location)
	if !isURL {
		log.Fatal("The location you provided is not a valid URL")
	}

	log.Printf("Starting web server... Location: %s, Port: %d", *location, *port)

	http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
		redirect(w, r, *location)
	})

	err := http.ListenAndServe(":"+strconv.Itoa(*port), nil)
	if err != nil {
		log.Fatal("Error starting web server. ", err)
	}
}

Usage:-

% ./redirection_service -h
Usage of ./redirection_service:
  -location string
        The URI at which the  (default "localhost:80")
  -port int
        Port at which the http server binds to (default 80)
% sudo ./redirection_service -location=https://abhijeetr.com
2016/01/15 09:33:54 Starting web server... Location: https://abhijeetr.com, Port: 80

Tuesday, January 12, 2016

Code Snippet: Fetch certificate information from a live endpoint in Go

I generally try to post snippets on my blog that I don't find on a Google search. This is like one of those snippets.

This sample code fetches the time to expiry for a certificate that's currently deployed at a live endpoint. 

Code:-

package main

import (
 "crypto/tls"
 "flag"
 "fmt"
 "time"
)

func main() {
 // Parse cmdline arguments using flag package
 server := flag.String("server", "abhijeetr.com", "Server to ping")
 port := flag.Uint("port", 443, "Port that has TLS")
 flag.Parse()

 conn, err := tls.Dial("tcp", fmt.Sprintf("%s:%d", *server, *port), &tls.Config{})
 if err != nil {
  panic("failed to connect: " + err.Error())
 }

 // Get the ConnectionState struct as that's the one which gives us x509.Certificate struct
 connectionState := conn.ConnectionState()

 fmt.Printf("Time to expiry for the certificate: %v\n", connectionState.PeerCertificates[0].NotAfter.Sub(time.Now()))
 conn.Close()
}

Usage:-

% ./ssl_certs -h
Usage of ./ssl_certs:
  -port uint
     Port that has TLS (default 443)
  -server string
     Server to ping (default "abhijeetr.com")
% ./ssl_certs
Time to expiry for the certificate: 1843h21m3.857177591s

Thursday, July 24, 2014

Accessing individual files from a "dd backup" of a encrypted hard disk/partition

Whenever I do something risky with my laptop like resizing partitions, I always do a full backup of the hard disk by running:
dd if=/dev/sda of=/path/to/backup/directory/image.img bs=4M
from a live archlinux USB pendrive.

Restoring from a backup like that is super easy.
dd if=/path/to/backup/directory/image.img of=/dev/sda bs=4M
Works perfectly. Every single time. Except licensing issues with Windows installation if you've a dual boot but WHO CARES!!!

In my case, one of my partitions /dev/sda5 is LUKS encrypted & if I want to access files from that partition using the .img file, things are not very straight forward but easy. For reference, this is what you'll typically do.
➜ 0 /home/shadyabhi [ 9:40PM] % sudo modprobe loop
➜ 0 /home/shadyabhi [ 9:40PM] % sudo losetup -v /dev/loop0 ./official_laptop_backup.img
➜ 0 /home/shadyabhi [ 9:40PM] % ls /dev/loop0*
/dev/loop0
➜ 0 /home/shadyabhi [ 9:40PM] % sudo partprobe /dev/loop0
➜ 0 /home/shadyabhi [ 9:40PM] % ls /dev/loop0*
/dev/loop0  /dev/loop0p1  /dev/loop0p2  /dev/loop0p5  /dev/loop0p6  /dev/loop0p7
➜ 0 /home/shadyabhi [ 9:40PM] % sudo mkdir /mnt/my_encrypted_partition
➜ 0 /home/shadyabhi [ 9:40PM] % sudo cryptsetup luksOpen /dev/loop0p5 encrypted_partition
Enter passphrase for /dev/loop0p5: 
➜ 0 /home/shadyabhi [ 9:40PM] % sudo mount /dev/mapper/encrypted_partition /mnt/my_encrypted_partition 
➜ 0 /home/shadyabhi [ 9:40PM] % sudo ls /mnt/my_encrypted_partition | wc -l
28
➜ 0 /home/shadyabhi [ 9:41PM] % 

Saturday, June 28, 2014

veth pair: How to know what interfaces are connected by a veth pair?

Here is a quick one. Couldn't find a direct solution on Google.

You can find out what's the peer of a interface by issuing the command:
[root@compute-1 ~]# sudo ethtool -S int-br-ex
NIC statistics:
     peer_ifindex: 55
[root@compute-1 ~]# ip link  | grep 55:
    link/ether c6:5f:55:82:20:72 brd ff:ff:ff:ff:ff:ff
55: phy-br-ex:  mtu 1500 qdisc pfifo_fast state UP qlen 1000
[root@compute-1 ~]#
So, you can easily find out that int-br-ex and phy-br-ex are peer to each other, that means they are connected via veth pair. To find out what all veth pairs are present in your system:
[root@compute-1 ~]# ip -d link show
.... output omitted ...
55: phy-br-ex:  mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:92:8e:3f:6e:f8 brd ff:ff:ff:ff:ff:ff
    veth 
56: int-br-ex:  mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether d6:3b:77:c0:70:4f brd ff:ff:ff:ff:ff:ff
    veth
[root@compute-1 ~]#
While using "-d" option, ip command gives you a little more details about the interface. In this case, it tells you whether a interface is part of a veth pair.