Tunnel browser traffic through ssh with limited network access

I go to conferences and meetups quite a lot, and one thing I always encounter on those trips, is bad wifi or internet access. The “public” networks made available to visitors often limit network access to http(s) and e-mail. This means for instance that I can’t use my XMPP/Jabber chat. A second problem with those networks is that my e-mail provider blocks e-mail access because they think I’m in an unknown location and I’m an intruder. The third problem is that there often are other people snooping around on the network trying to see what they can intercept. This is how I solve those problems for myself:

I use TorBrowser to set up a connection to the Tor network. This works on those limited networks, because Tor can connect over http(s). Tor also starts a SOCKS5 proxy on port 9150 on my computer (it does that by default), which I can use to tunnel traffic trough the Tor network. This would already solve most of my problems, except that my email provider doesn’t trust Tor, and won’t let me connect over the Tor network. We need to go deeper.

Another solution would be to use SSH with its -D option to setup a connection to my VPS (which my email provider trusts) and again, make a SOCKS5 proxy available to my local system to connect to the internet over that SSH connection. That, however, doesn’t work because those public wifi networks don’t allow SSH connections. Let’s combine this with the TorBrowser step: I setup SSH to use the proxy that TorBrowser started by putting this in my ~/.ssh/config file:

ProxyCommand /usr/bin/nc -x 127.0.0.1:9150 %h %p

Then, I use ssh over that proxy and let it expose another proxy, to which I can connect my email client:

ssh -D 9999 user@server

Now I can make all my apps (e-mail, XMPP/Jabber, etc…) use the SOCKS5 proxy on port 9999 to connect to the internet. They will appear to connect from my VPS, but actually, they’re going over SSH, which runs over Tor. It’s tricky, but it fixes my problem!

Enjoy it!

Mac OSX quick development setup

I recently changed jobs, and I thought it would be a time-saver in the future if i’d write down some quick instructions to setup my mac.

first step: install a decent browser

I use firefox. At the same time i installed some of my favorite plugins: Vimperator, JSONView, HTTPS Everywhere, Privacy Badger, Ghostery, Disconnect, Adblock Plus

install a decent terminal

I use iTerm2 all the time, it’s epic. Don’t forget to set the fullscreen mode to classic.

With a decent terminal in place, let’s get some of the things we need:

Install Homebrew
Run these commands to install some basic tools:
brew update
brew install ack chicken cowsay ctags ffmpeg git gpg guile newsbeuter node pandoc pass screen sl stow the_silver_searcher tig trash tree vim wget z

install dot files

Clone dot files from dotfiles repo on bitbucket and install them using stow.

install virtualbox & vagrant

Install virtualbox & vagrant

configure some stuff

• mails
• git
• dropbox?

That’s about it, set up in 1 hour!

Readline support in the GNU Guile REPL

As you all know, Scheme is epic, and guile is a really nice environment for scheme programming. What bothered me for some time was that the REPL (read-eval-print-loop) lacked readline support. After some digging around in the official guile documentation I found a really simpel way to activate it. Guile is actually compiled with readline support, you just have to enable it manually due to licencing problems.

just issue these commands at the REPL:
(use-modules (ice-9 readline))
(activate-readline)

I’ve gone one step further, and created a .guile config file in my home directory, containing those two lines. That enables readline automatically, every time i start the REPL. Nice!

Toon

IMAP email debugging

Isn’t it a problem when you have to debug a problem with emails in an existing project? You never know if they are sent and to whom? I just found out that python has an awesome IMAP debugger built in! Just set your project to use these email settings:

IMAP host: localhost
IMAP port: 1025
IMAP user: {leave empty}
IMAP password: {leave empty}

Then you run this in your terminal:

python -m smtpd -n -c DebuggingServer localhost:1025

You’ll now see every mail that’s sent from your application displayed in your terminal!

That’s it… Have fun debugging

How to set default screen resolution in Crunchbang GNU/Linux (#!)

1. Check your screen name

Open a terminal and type this command:
xrandr

You’ll get a list of available screen sizes, along with the name of your screen. In my case, that’s default.

2. Run xrandr as an OpenBox startup command

Open the ~/.config/openbox/autostart file, and add this line:
xrandr --output {screen name} --mode {window size} &

e.g.:
xrandr --output default --mode 1280x800 &

That’s it! OpenBox will now use your desired screen resolution at startup.

Track your dotfiles and homedir configurations in git using GNU Stow

I recently discovered the awesome GNU Stow application (works on unix-like systems like GNU/Linux or Mac OSX). Stow is a symlink manager, that allows you to easily deploy and remove files to or from a directory.

Setup

Let’s say you have some configuration files in your home directory (eg: .bashrc, .vimrc and a config directory .vim) and you want to have them in git to be able to track your changes and such. It’s not a really good idea to make your whole home directory a git repo. That’s where Stow comes in. Just create a configuration directory, for instance ‘dotfiles’, and create a subdirectory for every app you have configs for. Then place the appropriate files into the right subdirectory, like this:

/home
    /{user}
        /dotfiles
            /vim
                .vimrc
                /.vim
                    ...
            /bash
                .bashrc

You can now make the dotfiles directory a git repository, and keep your dotfiles safe in git. But they’re not yet in the right place, so we’ll ask our symlink manager to fix that for us.

Create symlinks

1. cd to the dotfiles directory
2. You can make Stow symlink the files to your homedir like so:
   stow {package}
   where you replace {package} with the name of the subdirectory you created earlier.
3. If you now want to remove a certain package’s config files, just do this:
   stow --delete {package}

How to install Stow

On Mac OSX

Use Homebrew:
brew install stow

On GNU/Linux

Install stow using your favorite package manager, e.g.:
apt-get install stow

That’s it!

Access the internet from a server without outbound internet access

Important

• This procedure assumes that you’re running a debian based OS, on your local machine. It might work from MacOSX too, possibly using Homebrew to install Privoxy, and from Windows, using the .exe installer for Privoxy and a *nix-like terminal like Mingw or Git Bash.
• The server OS needs to be unix based for this to work, and you’ll need root access over SSH.
• We will not have to install anything on the server.

How this works

To allow the server to access the internet, we’ll tunnel the server’s internet traffic through our local computer. To do this, we’ll need to run a simple proxy on our own computer. This proxy usually listens only for local connections. We’ll then port-forward the local port to a port on the server. From the server’s point of view, it then looks as if a proxy server is available on a local port. We can then tunnel certain server’s application’s traffic through that proxy.

Setting up the proxy over SSH

Install Privoxy proxy server and run it

On your local machine
sudo apt-get install privoxy
sudo service restart privoxy
Privoxy should now be running and accepting connections from localhost only, on port 8118.

Log in to the server over ssh and port-forward the privoxy port (8118) over that connection

ssh -R 8118:localhost:8118 root@{server}
This makes the server open port 8118 for connections, which will be forwarded to port 8118 on your local machine, on which privoxy will be listening. Privoxy will then handle the request.

Forwarding traffic over the Proxy

APT

Create or edit the /etc/apt/apt.conf file to set proxy settings for APT
On the server:
vim /etc/apt/apt.conf

Insert this line:
Acquire::http::Proxy "http://127.0.0.1:8118";

At this point, apt will work over the proxy.

The problem now is that we can’t resolve DNS requests over the proxy (We can’t use a SOCKS5 proxy because apt doesn’t work with SOCKS5 out of the box). To solve that problem we’ll edit the /etc/hosts file to contain the repository record.
On the server:
vim /etc/hosts

Insert this line (replace the x’s with the actual ip):
{xx.xx.xx.xx} archive.debian.org

If you don’t know how to get the ip address for archive.debian.org
On your local machine:
nslookup archive.debian.org

Now you should be able to use apt to update or install packages.
Example:
apt-get install git

Other programs

If you want to use the proxy for other programs on the server, like wget or git use this:
http_proxy=127.0.0.1:8118 {command}

This way you’re setting the proxy as an environment variable, while running the command {command}. Most (well-written) command line software will use that variable, but sometimes this won’t work.

Example:
http_proxy=127.0.0.1:8118 wget github.com

That’s it!

Tunnel web traffic while on a VPN without internet access

Whenever you need to access a server through a VPN for security reasons, you’ll most likely lose your internet connection.
If you still want or need to read your emails or browse the web for the solution of a problem, or get on IRC, it’s possible.

If there’s a server or any other computer with SSH access on your local network, it should still be reachable from your computer when it’s connected to the VPN. We’ll use that to our advantage by tunneling our web traffic through that second computer. The only problem is that DNS most likely won’t work anymore when you’re on the VPN, so we’ll have to know a login and the ip address of the second computer.

While connected to the vpn, we’ll create an ssh connection to the second computer, with some modifiers to do port-forwarding, effectively creating a SOCKS5 proxy.
ssh -C2qtnN -D 8081 {username}@{local-ip-address-of-2nd-computer}

This will not open a shell on the server, but it will make a socks5 proxy available on port 8081 (localhost).
You can now tell your browser to use that proxy as a SOCKS5 proxy, and access your mail, irc and other web needs through that connection.

If your DNS doesn’t get resolved, go to ‘about:config’ in your browser (firefox) and change
network.proxy.socks_remote_dns to boolean true

That’s it! 🙂

Execute a local sh file on a remote server over ssh

I wanted to automate running a deploy script on a few servers, just by running a command from my local command line. Executing a .sh file you have on your local filesystem on a remote computer, isn’t as easy as it looks. This is how i did it.

I run this line everytime i want to execute the script on the remote server (thanks to Yves for the updated version):

cat local-script.sh | ssh user@server "sed 's/{ctrl-v}{ctrl-m}//g' | bash"

What does this do:

cat local-script.sh
This just prints the local-script.sh file to the standardoutput. This file contains the script we want to execute on the server. In our case, the contents of the file gets piped to the next segment of the line by the pipe symbol (|)

ssh user@server
This part of the script creates an ssh connection to the server. This is pretty straightforward. When that’s done, everything between the double quotes (“) gets executed on the remote host.

sed 's/{ctrl-v}{ctrl-m}//g'
This part of the script converts carriage-returns into correct unix linefeeds. This is useful when you or someone in your team is working on a Windows machine.

| bash
Finally, it’s time to run the script, so we pipe (|) it to bash (the interpreter)

That’s it 🙂

Swap left ctrl and alt keys on Ubuntu and other distros

Lately, i’ve been working a lot on my ubuntu machine at work. For several reasons, i’ve wanted to swap the ctrl and alt keys:

1. I’m used to the OSX command key being right there where your thumb rests on the keyboard.
2. The pinky+index finger combination of ctrl+t or ctrl+v hurted my wrist

since i’m a heavy shortcut/keyboard user, the default ubuntu setup wasn’t working very well for me.

I found the solution here. Create a file in your home dir: ~/.Xmodmap and put this in it:

clear control
clear mod1
keycode 37 = Alt_L Meta_L
keycode 64 = Control_L
add control = Control_L Control_R
add mod1 = Alt_L Meta_L

It swaps the ctrl and alt keys, and makes your wrists and thumbs happy. There’s only one downside though. The ‘switch application’ key combination of ctrl+tab now becomes a little less handy… But that can be fixed too (keyboard settings > shortcuts).

Regards
Toon