This blog has moved!

If you are not automatically redirected to the new address, please direct your browser to
http://www.juxtaservices.com/blog/
and update your bookmarks.

Saturday, September 30, 2006

Wickedness And Perversion in Subversion

Oh the woe. The last week and a half or so has led to some interesting Subversion related discoveries, mostly shortcomings, defects, or lack of functionality. The follow is a solution focused rant about what went wrong and how I fixed it. Let the world learn from the mistakes of the past.

First off, I have to say that I really like Subversion (SVN). It is a huge step up from CVS, which was my first introduction to versioning software. With atomic checkins (all or nothing) and a simplified database storage method (well it's a DB by default), the file based CVS heap of revision numbers is left in the dust. Since commits are atomic, each commit can be assigned a single revision number, rather than each file having a separate version number based on the number of times it has been committed.

There are also some snazzy authentication methods, something that took a pretty sizable battle to do securely in CVS. The most notable of these is Apache integration using the dav_mod module. This allows repositories to be locked down via HTTP Authentication over SSL (HTTPS), while providing a secure web-based repository browser built right in! I was really loving life using Subversion, but as always, the technology gods find something to get angry over, and soon begin raining down the chaos. Whether it is my wanton disregard of DRM or the demeaning glare of my hacker goggles I will never know, but at any rate, I was bound to eventually uncover some issues, and alas, such was the case.

As many of you are aware, I lack nothing more than my thesis to complete my MS (which has been the case for the past year and a half). Well after ditching my first topic quite a ways in, I have finally settled on the one that will carry me accross the finish line. Sponsored by Doba, it involves the development of a web-based code deployment system to deploy development code to all the various servers it needs to be on to pass through a healthy deployment process. The core of this project involves some fairly intimate interaction with our repository software Subversion. As such, I have been working on that piece of the puzzle recently: trying to access SVN commands through a php script. Urban legend has it that there are some modules written to interact with the SVN client directly for PHP, but they are certainly still in the mythical stage at this point and definitely nto suited for mission-critical use. So, my only alternative was to encapsulate some direct system calls to the SVN client with a PHP object in order to utilize its functionality.

Anyways, on to the problems at hand. Below is a list of some of the "wickedness" I have encountered of late in trying to utilize SVN, along with the eventual solutions:

Externally Linked Folder Woes
So, since we utilize a fairly significant amount of base code, whose breadth spans multiple projects across the site, it is natural to want any changes to that code to be replicated accross its many locations. That in mind, its sort of a pain to have to make changes in several places to accomplish that. Therefore, one of the best solutions is to create externally linked folders in SVN. This allows the code to be stored in a single location, but included as part of various repository folders.

In comes trouble. This apparently has a few hiccups, at least in my version of SVN (which by the way I'm forced to get through redhat, so it is a bit outdated from the most recent version). It was noticed, that in order for the working copy to recognize these new externally linked folders, you have to make it jump through a few hoops and do a little dance. First of all, the root folder of the project (in most cases the 'trunk' folder) has to be updated. Well I didn't want to update everything inside it as well, so I was forced to use the -N flag to make it non-recursive. This alone did not do the trick however. I also had to actually copy the new folders into the path in the working copy. Once I did that I was able to call svn update on the linked folders and they appeared in the SVN versioning as if they had always been there. Not sure why that was the case, but I'm told that is fixed in later versions.

Certificate Hostname Mismatch Woes
This is one of the stupidest issues I have encountered as of yet in SVN. As part of my automated deployment tool, I must execute subversion commands from a web script, which runs as the apache user. For security reasons, this user does not have any home folder, shell functionality, or other resources that conventional users do. Subversion commands however, will present an accept dialog for any less than perfect security problems they encounter, which since the web user has no shell functionality, it is impossible to connect as them and permanently accept the certificate.

Here comes the really stupid part. There used to be command flag in the earlier subversion clients that allowed you to ignore certificate warnings and accept them automatically, but it was removed because it was deemed obsolete! Idiotic decision. That means that either the certificate must be perfect (in this case it was self-signed and I was originally getting a warning because the hostname did not match, and because the CA was not trusted), or you must add the CA cert to the list of approved CA's for all clients on the server if you are going to connect through a web script. Not very friendly towards dynamically allowing the addition of new SVN servers.

So to solve this problem, I finally just had to create a new certificate with the proper hostname to solve the hostname warning, and then add the certificate's .crt file to the server hosting the web script, so that it's SVN client would ignore the CA warning error. I won't go through how to create a new certificate, since there are lots of good sites on that (try http://slacksite.com/apache/certificate.html), but I will tell you how to add the .crt file to the list of approved CA's.

To rid yourself of an invalid CA certificate error. on the SVN client server, go to /etc/subversion as root (if the folder does not exist, jsuts create it - I spent a while figuring out you could do that even with older versions of SVN). Copy the .crt file that was used to sign the offending certificate into this folder. Make sure the certificate file is world readable. Now create a file called 'servers' in the same folder (/etc/subversion), or edit the one that is already there. For added security, we only want to accept the CA for the URL of the SVN repository. So, to the 'servers' file, make the following addition (where 'mygroupname' is a name you have chosen for the group, 'host.domain' is the hostname of the server hosting the SVN repository, and 'host.domain.crt' is the name of the certificate file that you copied over):

[groups]
mygroupname = host.domain

[mygroupname]
ssl-authority-files=/etc/subversion/host.domain.crt

Now, when you connect to the repository on the specified server you should no longer get either of those certificate error warnings.

Subversion HTTP Passwords Stored in Plaintext
The final piece of subversion perversion that I want to share has to do with the fact that HTTP passwords used to connect to an SSL encrypted repository URL, are stored in plaintext on the client machine! Amazing! I know that HTTP authentication is passed in plaintext, but why would anyone use that without tunnelling it through SSL! That's even how it is recommended in the SVN documentation! So, what I don't understand, is why the client wasn't written to encrypt, or at least hash those passwords when caching them on the client. If you want to taste of what I am talking about, connect your SVN client to an HTTP or HTTPS repository location, and then check out '/home/username/.subversion/auth/svn.simple'. Do a 'cat' on one of the files in that folder and you will see the username and password plainly shown! I'm sorry, but that just amazes me as to why people still do that in this day and age!

Monday, September 18, 2006

SSH in Yo' Face!

I recently encountered a rather annoying little caveat of using SSH public key authentication. I have some scripts that use public key authentication to connect to other servers and accomplish various tasks, and since they all run within an internal network, it suits me just fine. However, the other day (it appears due to a mistyped rsync command) the permissions for the user's home folder on the server I was connecting to got altered to be world read and writable (777). The following few hours were spent trying to figure out what was going on.

The main symptom was that I was able to connect to the destination server using password authentication, but for some reason, the public key authentication was not working. I went through verifying that the authorized_keys file contained the correct key for the connecting host, and even generated a new key and tried to use that. After spending a few hours fighting the battle, I had finally run out of ideas so I gave it a few days to rest. Today, upon taking a look at the problem a buddy of mine pointed out the offending file permissions. For some strange reason, SSH does not like it when the home directory has world writeable permissions. All that wnd we were back in business. It's kind of counter-intuitive if you ask me, but I guess that prevents the user from leaving sensitive keys open to being read by any user. I may not have never figured that out, so props to Andy for being so freakin' smart!

Sunday, September 17, 2006

What's My IP? What's Google's IP?

So it's been a while since I've posted (things have been pretty hectic). I've actually got another fairly lengthy post started about some recent issues with Subversion, but it's not done yet, so it will have to wait a bit. However, I couldn't help but take a moment to post about my recent discovery of a few moments ago.


I was working on setting up some dynamic DNS service through dyndns.org (which is free by the way), and so I was trying to figure out my external IP. After going to www.whatsmyip.com, and discovering it was some crap URL squatting site, I then decided to google "what's my ip". Aside from discovering the real URL was www.whatsmyip.org, in the search results I also noticed that the descriptions on some of the related sites disclosed the IP address (66.249.65.231) of Google's search spider (http://www.google.com/search?q=whats+my+ip&start=0&ie=utf-8&oe=utf-8&client=firefox-a&rls=org.mozilla:en-US:official)!! Over a period of time, if done each time Google updates its search database (which according to my familiar source who works there, it can be anywhere from daily to every few weeks), you could begin to develop a list of their spidering IPs.

Since I'm a relatively respectful person when it comes to online activity (and my online presence is pretty freakin' small compared to the big G), this probably won't really be of any use to me. I'm sure you could also dig around and find a published list of their spidering IPs. Google I imagine is also pretty respectful when it comes to the standard robots.txt file and other spidering deterrents that sites can employ, but that doesn't mean everyone is. So I tried the same trick with several other search engines and the same result occurred. Yahoo!'s last search was coming from 72.30.215.200 (http://search.yahoo.com/search?p=what%27s+my+ip&fr=yfp-t-500&toggle=1&cop=&ei=UTF-8). AskGeeves was 65.214.44.193 (http://www.ask.com/web?q=what%27s+my+ip&o=0&l=dir&qsrc=0&qid=1F618F69063F9E097B55B544E303F854&page=2). MSN was 65.54.188.67 (http://search.msn.com/results.aspx?q=what%27s+my+ip&FORM=MSNH). The list would undoubtedly go on. Employing this tactic on a less respectable search engine site may indeed reveal some info on which IP's they are using to harass the rest of the Internet.

Friday, September 01, 2006

Pack Me a Juice Box Ma! Actually, make it 6!



So alot of you that know me (I'll call you my "peeps" ), know that I am kind of a sucker for a bargain. As such, one of my regular stops on my daily tour of the world wide web is slickdeals.net. It's basically a forum site where people post great deals and amazing bargains. Despite some really good deals I seldom really find justification for buying stuff (alot of times because there are rebates involved, and sometimes even some sketchy sites that make me wary of providing credit card info), but the other day I was scanning the daily highlights and noticed a really awesome deal from overstock.com.

Target bargain: the Mattel Juice Box. Bargain price: $25.99 - $5.00 (with coupon code) + $1.00 shipping = $21.99 for a pack of 6. That's only $3.67 a piece! Now that's a bargain if I've ever seen one!

What's so great about the Juice Box? Well, it was put out by Mattel as a personal media player for kids and such. You were supposed to buy new media on these little cartridges, but it sorta fizzled out, and for the last year or so, retailers have been dumping their stock. I have been hearing about them for a while, but have never been lucky enough to find any on sale. Until now.

Oh, I could care less about having something cheap to watch little kiddie cartoons on (although my boy will probably end up with one of them). No, the real gem here is the fact that they are quite hackable. It runs under a version of uclinux, so its quite feasible to load a custom uclinux version onto it for creating things like an mp3 player, video player (albeit crappy frame-rate-wise), or even an LCD picture frame. They can even fairly easily (it seems) do some custom soldering to get them to accept SD and MMC memory cards directly. I have always been a fan of tinkering, and once I found it was quick to jump on the hardware hacking bandwagon, so this project was right up my alley (one of my favorite hardware hacking sites is www.hackaday.com). Unfortunately, until my thesis is finished, I am a bit limited on spare time for tinkering, but this hasn't stopped me from amassing a few projects in the queue. So, stay tuned in some future post for when I actually show you that I've been able to do something cool with my 6 shiny new Juice Boxes. In the meantime, below are a few of the links I've dug up so far for Juice Box projects:

Juice Box Linux Hacker forum
Juice Box Linux wiki
Juice Box Messenger Bag (on hackaday)
Juice Box eBook Reader


There are surely many more out there. I have thoughts of maybe adding a GPS receiver into one of them, or using it for text-to-speech. I guess it will all depend on how well it runs uclinix. Until then....