Ok, so I wanted to cover packing binary files with my students in Systems Programming. Why you ask? It's good for them for starters. Until you read a packed binary specification and write a packer/unpacker, I don't think you truly understand what the low level systems calls like open(), read(), and write() are capable of. Plus, I wanted to foreshadow what we will be covering when we get to networking and we cover binary network protocols.
So I knew I wanted the students to either pack or unpack some binary format, but I struggled with what to have them do. Most of the students have had very little practical programming experience, even though they're all basically either seniors in computer science or masters degree students in computer science. I know, that seems ridiculous, and it is, but that's a topic for another post. So I started looking around. I thought it'd be cool to extract exif data from a jpeg. So I started reading up on it, and saw that it wasn't trivial. It's probably something seniors and grad students SHOULD be able to handle, but I didn't want to push it. They need to walk before they can run. Then I thought, tar...tar is pretty simple and it's been around forever and it's well documented. So I looked at tar. While it isn't THAT bad, it still seemed a bit steep for their first time out.
So, I ended up developing an extremely simple file format. It's in the same vein as tar, but much much much (did I say much) simpler. Basically it starts with a 4 byte header to declare that it's the format I came up with for the class, followed by variable length records for each file. The "variable length records" start with a 2 byte short declaring the length of the filename, followed by that many bytes which is the filename. That is followed by 8 bytes representing a 64bit int, which specified the number of bytes that file is, immediately followed by the bytes for the file. The next record follows immediately after. So yeah, super simple. No support for directory structures or anything...just file names and file data. Hopefully, this won't be too difficult.
Ok, so I know I mentioned a financial motivation. Among the miscellaneous files packed into the sample I plan to give the students, is a wallet.dat file. For those of you familiar with cryptocurrency, you know what that means. For those of you who are not familiar, it is basically where the encryption keys are stored for a "wallet". Basically, if you have the wallet.dat file and it's not encrypted, then you can spend any of the coins associated with any of the account numbers in the wallet. And yes, I put some in there. Now mind you, this isn't a bitcoin wallet, this is a dogecoin wallet. So the coins in there are not quite as valuable currently, but that's not really the point. The main point is, only one person can control a wallet. The first student who finds the wallet.dat, and realizes what it is, can transfer all the coins out of it to another wallet. Any subsequent student who looks at the wallet, it'll be useless. So it's sort of an arms race. More importantly, I hope for it to show me which of my students are the most committed and determined.
Oh, and due to the awesomeness of crypto currency, if anyone reads this who has a few dogecoin to spare, you can send them to:
DHHCc6kfpMCLE4FEaSaPGCTMpbNUWRmJi8
It'll be interesting to see if one of my students actually gets the wallet out and takes the coins. I'm also debating just not telling them and see if one of them figures it out on their own. Of course, the file is just named wallet.dat and so without knowing that it is a dogecoin specific wallet, it may be complicated. But I did include a picture of a shibe...so that should be a clue right?
I'll hopefully follow up after the assignment has been submitted by everyone.
Friday, March 14, 2014
Monday, February 24, 2014
SSH just makes you feel like a freakin' ninja
Ok, so this is not a new thing, I've done this many times before, but every time I use SSH to solve some odd problem, I feel like I'm way more awesome than I really am. I think it must be how the cool kids in high school felt all the time.
/etc/network/interfaces
Made it look something along those lines, and rebooted the machine, crossed my fingers and waited for it to come back up. Luckily it did, because if it didn't, I'd be driving out late at night, and that didn't sound fun to me.
Now, you just use SSH and tell it to use 'ethernet' tunnel devices (i.e. tap devices):
Once you do that, you should be logged into the remote machine, and it should have an extra tap0 device (or some other number if you already have tap devices defined). At this point, you simply bring up the tap0 device on both ends, and add the device to your bridge. So something like:
Run that on both ends, replacing tap0 with the tap device that was created, and br0 with the name of the bridge device you created. At this point, you now have a bridge between your local spare ethernet device, and your remote network. Essentially, anything you connect to your local spare ethernet device will act as if it was directly plugged into the remote network!
So what I did, was connect up a small 10/100 switch to the spare ethernet port, so now this 10/100 switch is essentially bridged to the switch at the remote network. Pretty cool. Now I just plug in my SIP phone, power it on, and what do you know, it gets a dhcp address from the remote network, and registers itself with the phone system at my client's location. At this point, I can test the phone system at my leisure just like I was physically on their premises.
But every time I use SSH to pull off something cool like that, it just feels awesome. I just wish all sysadmin tasks were as exciting. I think it's more that I over came the challenge of testing something without having to physically go there. Either way, it's cool, at least I think so.
So, again, I needed to test something on a customer's network, and I still don't have a dedicated VPN. I keep thinking I'll set one up, but I just don't need to use it that often. And usually, the only thing I'd want a dedicated VPN for, is to connect a physical device to their network from home. In order to do that, I'd need like a dedicated VPN box and a dedicated vlan for traffic to each customer setup like that. But I wouldn't have them connected 24/7, so really, it just doesn't make sense. It's much easier to just create an adhoc VPN with ssh when needed.
So for the un-initiated, openssh for quite a while now, has had support for creating virtual network adapters (tun and tap) on both sides of an ssh connection, and all traffic between the two virtual adapters are tunneled over your ssh connection. It's pretty slick. You can either create a point-to-point connection (tun device) or virtual ethernet (tap device). The latter lets you create a bridged network, which is what I tend to do.
So, I had a customer with an asterisk phone system issue. Their handsets weren't configured properly, and DTMF signals were either not being sent, or being interpreted incorrectly. But I needed a handset like theirs, on their network, connected to their phone system to test out the various settings to find the right combination. I really didn't want to drive in. Luckily, I had one of their spare handsets at my house. Now all I needed was a way to make this wired phone think it was physically on their network. SSH to the rescue.
Now, this isn't a setup that you can pull off easily. There are many caveats. For example, to created an ethernet bridge, you need machines with SSH on both sides, that also have an ethernet adapter setup in bridge mode. Luckily, I have an embedded debian box on their premise just for SSH'ing into. I just had to change it's network config to create a bridge device, and add it's eth0 to the bridge on startup, then reboot and hope i didn't screw up. Luckily, this time, I didn't screw up, and I was able to get back in after reboot.
So here's my setup, I had a debian desktop at my house with a spare ethernet adapter (I've also used a laptop with a usb to ethernet adapter in the past). I set up the spare ethernet adapter (eth1) to be part of a bridge.
Next, I made sure the other end was set up similarly, I simply edited it's
ifconfig eth1 0.0.0.0 promisc up
brctl addbr br0
brctl addif br0 eth1
/etc/network/interfaces
iface eth0 inet manual
auto br0
iface br0 inet static
bridge_ports eth0
address 192.168.1.42
netmask 255.255.255.0
gateway 192.168.1.254
Made it look something along those lines, and rebooted the machine, crossed my fingers and waited for it to come back up. Luckily it did, because if it didn't, I'd be driving out late at night, and that didn't sound fun to me.
Now, you just use SSH and tell it to use 'ethernet' tunnel devices (i.e. tap devices):
sudo ssh root@REMOTE -o Tunnel=ethernet -w any:anyA couple things to note above. First of all, notice I'm both doing sudo, and also ssh'ing to the remote hosts root account. This is because it seems that only root can create the necessary tun/tap devices on either end. The next thing to note, I had to specify the extra -o option before the -w option. If the -w option came first in the command line, it created tun devices on both ends instead of tap devices (tun can only do point to point networks, i.e. routed networks, I want bridged networks).
Once you do that, you should be logged into the remote machine, and it should have an extra tap0 device (or some other number if you already have tap devices defined). At this point, you simply bring up the tap0 device on both ends, and add the device to your bridge. So something like:
ifconfig tap0 0.0.0.0 promisc up
brctl addif br0 tap0
Run that on both ends, replacing tap0 with the tap device that was created, and br0 with the name of the bridge device you created. At this point, you now have a bridge between your local spare ethernet device, and your remote network. Essentially, anything you connect to your local spare ethernet device will act as if it was directly plugged into the remote network!
So what I did, was connect up a small 10/100 switch to the spare ethernet port, so now this 10/100 switch is essentially bridged to the switch at the remote network. Pretty cool. Now I just plug in my SIP phone, power it on, and what do you know, it gets a dhcp address from the remote network, and registers itself with the phone system at my client's location. At this point, I can test the phone system at my leisure just like I was physically on their premises.
But every time I use SSH to pull off something cool like that, it just feels awesome. I just wish all sysadmin tasks were as exciting. I think it's more that I over came the challenge of testing something without having to physically go there. Either way, it's cool, at least I think so.
Thursday, February 20, 2014
Love ANSI C, except when it comes to strings!
So, in class the other night, we wrote up a very simplified version of ls. I was just trying to demonstrate to the students what is involved in opening a directory and reading directory entry structures out. I figured ls would be the simplest example of doing that. Well, I got the idea, lets take it a bit further than just printing file names, and print out some info from the stat structure for each file as well.
However, to call stat, you need to pass in a full path, but we only have a filename. I really didn't feel like doing a bunch of string manipulation in C in class, so I tried to see if we could get the stat info by some how using the inode number. I couldn't come up with a solution quickly and the class was basically over, so I told the students I we would come back to it next time.
So I kept researching. I was convinced there should be a way to get the same info stat returns by just using the inode number. I mean the inode is where that information is stored for crying out loud. So after wasting part of my life searching, I found some answers that basically said the POSIX standard won't let you use inode number directly for anything because it's not portable. Basically, some systems may not even implement their filesystems using inodes. So that wouldn't be a portable solution, and as we all know, the whole point of POSIX is to make code portable. So the only reliable way to get stat info, is to give it a path. So I realized I was stuck having to do string manipulation.
After giving in to the fact that I wouldn't be able to avoid using string manipulation, I started writing the code. Man, I forgot how painful it is to do something as simple as joining two strings together in C. I had written a function to take a path, and list the files in the directory. It was a very simple function. Keyword being was. After adding a bunch of code to create an appropriately sized temporary char array to store my temp string in, and then all the code to concatenate the strings and make sure there were path separators, etc, the function size just ballooned. Most of the code, probably more than half, was just for manipulating strings.
Anyway, I'm not saying I hate it, just that it's a bit painful. After mostly using python for years, you REALLY take for granted how easy simple string manipulation is.
Ok, that is all for now, just had to vent.
However, to call stat, you need to pass in a full path, but we only have a filename. I really didn't feel like doing a bunch of string manipulation in C in class, so I tried to see if we could get the stat info by some how using the inode number. I couldn't come up with a solution quickly and the class was basically over, so I told the students I we would come back to it next time.
So I kept researching. I was convinced there should be a way to get the same info stat returns by just using the inode number. I mean the inode is where that information is stored for crying out loud. So after wasting part of my life searching, I found some answers that basically said the POSIX standard won't let you use inode number directly for anything because it's not portable. Basically, some systems may not even implement their filesystems using inodes. So that wouldn't be a portable solution, and as we all know, the whole point of POSIX is to make code portable. So the only reliable way to get stat info, is to give it a path. So I realized I was stuck having to do string manipulation.
After giving in to the fact that I wouldn't be able to avoid using string manipulation, I started writing the code. Man, I forgot how painful it is to do something as simple as joining two strings together in C. I had written a function to take a path, and list the files in the directory. It was a very simple function. Keyword being was. After adding a bunch of code to create an appropriately sized temporary char array to store my temp string in, and then all the code to concatenate the strings and make sure there were path separators, etc, the function size just ballooned. Most of the code, probably more than half, was just for manipulating strings.
Anyway, I'm not saying I hate it, just that it's a bit painful. After mostly using python for years, you REALLY take for granted how easy simple string manipulation is.
Ok, that is all for now, just had to vent.
Sunday, February 16, 2014
Initial Commit
My initial commit on the new blog. I decided to just go with blogger. I really wanted to (and almost did) write up a simple blogging tool to generate static html. In fact, I do this for my teaching websites. I was tempted to just copy my code from one of them, and go with just a clone of it for my personal site. But then I got to debating with myself. Then began the flip flopping. You know, well it'd be cool to just write up some markdown text posts and store the blog in git. But then what if I get the urge to blog and all I have is my phone. But I want to maintain all my content and don't want to give it to someone else.
So, in the end, I gave up, and just went with blogger. It has a decent android app, works great on the chromebook, I can download all my content and move it elsewhere if I choose, etc. Besides, I can always use my new favorite static website generating tools (frozen flask and flask flat pages) for plenty of other projects.
I fully intend (now that I can easily write blog posts from anywhere, even my phone), to write much much more. But, that's what all my friends say when they start blogging again, and we all know how long that usually lasts.
Well, stay tuned just in case.
Subscribe to:
Posts (Atom)