The Cranky Sysadmin A world of technology, fun, and ignorant rants.

March 19, 2010

Hadoop? What kind of a name is Hadoop?

Filed under: Programming,System Administration — Cranky Sysadmin @ 8:03 am

Briefly, as I understand it, hadoop is a distributed program which allows one to aggregate data by processing across a bunch (more than 2) of computers. The name apparently comes from the name the lead developer’s son gave to his stuffed toy elephant. It’s not just the name that I dislike though.
Uninformed opinions and rants here

December 7, 2009

Steps Closer to mod_js for Apache

Filed under: Programming — Cranky Sysadmin @ 7:35 pm

I started out thinking that I’d learn JavaScript so that I could make AJAX games. I am ending up in the land of C programming while trying to get a functional mod_js running.
Holy crap! mod_js!

Toward Server-side JavaScript in Apache.

Filed under: Programming — Cranky Sysadmin @ 8:10 am

I’ve always wondered why no one ever created a JavaScript module for apache. Many other languages have apache bindings, but not JavaScript. Being able to program in the same language client-side and server-side would seem to be desirable.
Read on about server-side JavaScript

December 6, 2009

I don’t have ADD, I really don’t!

Filed under: Games,Programming — Cranky Sysadmin @ 7:00 pm

The other day I got home, finished dinner, and rushed to my office to play either Perfect World or Runes of Magic. After sitting there for a few minutes, I couldn’t log in. There was no problem with my account. My brain was refusing to allow me to play. I haven’t logged in for more then 2 minutes since then. I now seem to have some spare time. Some will be used for my wife and some, I’ll use to finally learn javascript and maybe finish that mod_js project I started working on before I got lost in MMO-land. Some day (maybe soon) I’ll probably get back to MMO’s, but for now, I’m burned out on them and I should do something else.

January 5, 2009

The Cost of Checks

Filed under: Programming,System Administration — Cranky Sysadmin @ 10:14 am

I read an interesting article by Paul Graham about the cost of checks in an organization. Checks in this case refer to things like comprehensive QA of a product or feature before release. Paul’s premise is that all checks have a cost and some of the costs are surprisingly high. I tend to agree, but I also see things through the lens of an operations guy. When I release a product that has gone through no checks (I have been told to do this), there is a high probability that something will break. In many cases, the breakage can be crippling. I think there is a good chance that some checks will actually make a company more nimble, especially as a code base grows in complexity and more people depend on your product.

If a company spends as much or more time fixing the problems caused by a release as they spent on the release, maybe some checks need to be put in place to cut down that wasted effort. The problem is finding a balance where the check costs less then this extra effort and pain.

Maybe unit tests help. People who do unit tests seem to have different opinions about how effective they are. Some say that it causes them to take twice as long to write the code. Others say it helps them write the code faster since they have a codified requirement.

Automated testing is another thing that can make the release process better. There is a large cost associated with this in many cases though. You need a highly qualified QA guy who knows how to program and knows the product to write the tests. If he’s that good, maybe his time is better spent helping build the product (if this is a small organization). Could the automated tests be written by the group who develops the software? Sure, but it will slow down “real” development work.

I think the real problem is that as complexity increases, the need for checks increases. Eventually, one gets to a point where progress is ponderously slow. What do you do about this? Well, all of the solutions that I know of are bad in some way. One has to make a choice between pain and searing pain.

  • Keep it simple. Simple systems are easier to check and there is less risk in changing them since they’re easily understood.
  • Keep it small. This is really a subset of simple. If you have 5 simple systems (like web servers), you’ll have an easier time managing them then if you have 100 simple systems.
  • If you can’t keep it simple, only make it as complex as it has to be. This means managing customer expectations (which no one wants to do in my experience). Make sure the system isn’t more complex then your developers and operations staff can manage.
  • Don’t let complexity creep up on you. Know that it’s coming and plan for it. Know that your costs will rise as the complexity rises. Find ways to make the cost rise as slowly as possible.
  • If you have to have complexity, automate the heck out of everything you can find. This is easy to say, but if the system has grown so complex that all you do is fight fires, then you won’t have time to automate unless you are willing to accept a lower level of service for the time it takes to automate.

I am learning these things the hard way, so I don’t know of any elegant solutions to the problem of complexity which go beyond what I’ve already mentioned. Maybe after a few more startups I’ll have more useful advice then, “Watch out! You’re headed for a big bucket o’ misery!”

June 8, 2008

That Which is Measured…

Filed under: Navel Gazing,Programming,System Administration,World of Warcraft — Cranky Sysadmin @ 11:59 am

Lately I’ve been doing a lot of pickup groups in World of Warcraft. If we’re doing something hard, those that ask what my healing bonus is seem to do the best. If no one asks me about it, the outing almost always ends in disaster. In my work experience, small companies which measure their performance and their team’s performance seem to do better then those which are fast and loose with metrics.

The Hawthorne Effect describes a situation where merely measuring workers improves their performance for a short period of time. I think something similar, but better is going on with WoW pickup groups and small companies. If you’re willing to measure yourself, you’re more willing to learn to perform better in fundamental and permanent ways. Can measurements be gamed? Maybe they can. In WoW, I will forgo other stats to bring my healing bonus up. This is probably not great for my long term raiding viability, but it is the one thing that other non-healers know how to measure. Fortunately, in WoW, there are ways to cover up the slack in some of my stats. I can eat buff foods, and drink elixirs to cover up my woefully low stamina for instance. For me, this leaves a guilty voice in my head that says I’m not measuring all of the stuff I need to, so yes I’m beginning to look for gear with more stamina and mp5 so I can stay alive and contribute longer.

In real life, I hate being measured and measuring myself. I’m a fat-body because I don’t like to get on the scale. I’m not a great programmer because (among other reasons) I’m not willing to profile my work and measure it against the good works out there, or against any performance goal. I just want to slam in the code and pray that it works. As a sysadmin, the idea of measuring my work by the number of warnings and alarms that nagios produces annoys me, but if I make good alarms, they are actually a passable measure of how stable the system is.

So what do I have to do? I have to think briefly about whether I’m measuring the right things about myself and then start measuring.

November 28, 2007

Analysing Apache Web server response times

Filed under: Programming,System Administration — Cranky Sysadmin @ 11:51 am

Apache logs a lot of useful information about each request in the access logs. By default, it doesn’t include response times for each request, but that’s easy to remedy. In your httpd.conf, find the line that looks like this:

LogFormat “%h %l %u %t \”%r\” %>s %b \”%{Referer}i\” \”%{User-Agent}i\”” combined

and replace it with:

LogFormat “%h %l %u %t \”%r\” %>s %b \”%{Referer}i\” \”%{User-Agent}i\” %D” combined

The %D appends the response time to the end of the log line. After you have collected some hits with response times, you can use a perl one-liner to look at timestamps and response times like so:

tail -1000 access_log|perl -ne ‘chomp; split; $_[3] =~ s/^\[//; print “$_[3] $_[$#_]\n”;’

replace tail -1000 with whatever number of log lines you want to look at. I use some GNU date magic,  perl’s GD:Graph, and some shell programming to build a graph of average and high response times over the last hour. For now, I leave this as an exercise for the reader. GD::Graph is a great tool for visualizing all kinds of data where rrdtool may not suffice.

October 25, 2007

Tools for web programming

Filed under: Programming — Cranky Sysadmin @ 2:10 pm

I’ve been thinking about web startups lately. My aim is to build a successful one someday. That got me thinking about all of the tools out there that can give a potential founder a leg up. Let’s narrow the field arbitrarily. I want to use free tools to minimize financial risk, should the startup tank. If I can’t have free, then I want cheap and widely available. I also want tools that are flexible and those which I can look at the inner workings. You guessed it, I’ll be looking at a lot of open source software.

Let’s take a look at some stacks that I would choose:

  • Linux, Apache, mod_perl, PostgreSQL.
  • Linux, Apache, php, PostgreSQL (or mySQL).
  • Linux, Lisp, and some database… or none.

Why Linux? Well, Linux is sufficient. Of the free unices, Linux is the most popular and widespread. Linux has good driver support.

Why Apache? Apache is the most featureful open source web server out there. Apache has a great API if you want to build components in C. Most popular languages in the open source world have an Apache API.

If I have to use a database, I prefer to use PostgreSQL. MySQL has come a long way in recent versions, but PostgreSQL still has some features that I think I need for web programming. One example is the network types. I can do somthing like this, select * from networks where “10.10.0.0/16” >> ip_address; . The >> means “contains”. In mySQL, or Oracle, I have to build a table of networks and/or IP addresses, iterate through them, and perform the network math myself. PostgreSQL also has a mature procedural language, and if I don’t like pgpl, I can use perl… or tcl… or python as my procedural language.

For familiarity, I would probably choose perl as my language of choice. PHP is also a strong candidate, but for reasons that I will detail in a later article, I probably would shy away from PHP. Perl has very tight integration with Apache. You can perform amazing gymnastics with the life cycle of an http request in mod_perl. Perl also has the advantage of CPAN, a repository of perl modules that do almost everything under the sun. One has to be careful about using CPAN as the quality of the modules vary.

I’d love to use lisp someday in a web application, but only if I have all of the time I need to learn its advanced features. I can imagine a lot of uses for continuations for instance, but I don’t understand them well enough to make effective use of them. So, I’ll keep plugging away at my lisp books and some day, I’ll write a web app in lisp.

Obviously, other people will make different choices based on their experience. Are there technologies that I wouldn’t use if I could avoid them? Yes, there are.

  • Windows: Everything you need to use with windows costs extra: office tools, dev tools, database, mail server… For reliability, I’ve never had good experiences with Windows. The company I work for has had production linux servers for 9 months and none of them have ever been rebooted. It’s rare that I find a windows admin that can say that… In fact, I’ve never heard a windows admin say that.
  • Java/Jboss: From a sysadmin perspective, I’ve had bad experiences with Java/Jboss. It has a tendency to crash. Application startup is long to “holy crap, that’s long”.  The performance is variable due to garbage collection. Configuring a JBOSS instance requires a lot of specialized knowledge. Jboss programmers tell me that it has a lot of conveniences, but I’ve seen a lot of work go into fixing some of the problems caused by the Jboss infrastructure. As far as Java goes, I’m just not a big fan of the language. It feels like I have to type way too much to get anything done.
  • Oracle: It’s a very featureful database, but it’s also very expensive to put into production, and it doesn’t perform significantly better for general database tasks then PostgreSQL or MySQL. Oracle also *requires* an Oracle DBA if you want a smooth running system in a fast changing environment like a startup. PostgreSQL and MySQL don’t require this kind of attention.

Well, that’s my navel gazing for the day. Next time, I’ll talk about my World of Warcraft adventures and the fact that my wife thinks (probably correctly) that I’m addicted.

October 22, 2007

Hello World

Filed under: Games,Navel Gazing,Politics,Programming,System Administration — Cranky Sysadmin @ 10:29 am

Welcome to my corner of the world. I’ll try to add fresh content once in a while on topics that interest me. If you’re fortunate, some of the topics may interest you. If they don’t, there is always google (my interests are subject to change and may shift without notice).

Currently the interests that I am willing to rant about include; Computer Programming, System Administration, Computer Games, Politics (or meta politics), and some random philosophical navel gazing.

« Newer Posts

Powered by WordPress