The Unix Philosophy

For the past couple of weeks I have been doing some research on best practices for software development. Along the way, I came across the term Unix Philosophy. Its basically a set of design decisions and practices adopted by the architects of Unix in early days. Whether Unix actually implements these practices is questionable, but they are definitely worth exploring.

Unix and other Operating Systems

The Unix philosophy is clearly very different from the philosophy behind the Microsoft Windows operating systems. The authors on Unix assumed that its users will be computer literates. Hence there primary focus was on robustness, maintainability and extensibility of  code modules; rather than on user experience. On the other hand, Microsoft Windows and other commercial Operating Systems are governed by a very different set of priorities. That is, the emphasis is on gaining or maintaining market share and maximizing profits. As a result, these systems often generate very large, monolithic (lack of modularity), complex and closely guarded source code.

What are the philosophies?

The most prominent philosophy guiding Unix is the concept of modularity. Doug McIlroy, the inventor of Unix pipes and one of the founders of the Unix tradition, elaborates this concept: Design programs to do only a single thing, but to do it well, and to work together well with other programs.

The Art of Unix Programming by Eric Steven Raymond provides an excellent discussion on the historical and philosophical aspects of Unix. I found the Chapter 1 of the book absolutely fascinating to read. From there I am posting the 17 rules of Unix design abstracted by the author.

1. Rule of Modularity: Write simple parts connected by clean interfaces.

2. Rule of Clarity: Clarity is better than cleverness.

3. Rule of Composition: Design programs to be connected to other programs.

4. Rule of Separation: Separate policy from mechanism; separate interfaces from engines.

5. Rule of Simplicity: Design for simplicity; add complexity only where you must.

6. Rule of Parsimony: Write a big program only when it is clear by demonstration that nothing else will do.

7. Rule of Transparency: Design for visibility to make inspection and debugging easier.

8. Rule of Robustness: Robustness is the child of transparency and simplicity.

9. Rule of Representation: Fold knowledge into data so program logic can be stupid and robust.

10. Rule of Least Surprise: In interface design, always do the least surprising thing.

11. Rule of Silence: When a program has nothing surprising to say, it should say nothing.

12. Rule of Repair: When you must fail, fail noisily and as soon as possible.

13. Rule of Economy: Programmer time is expensive; conserve it in preference to machine time.

14. Rule of Generation: Avoid hand-hacking; write programs to write programs when you can.

15. Rule of Optimization: Prototype before polishing. Get it working before you optimize it.

16. Rule of Diversity: Distrust all claims for “one true way”.

17. Rule of Extensibility: Design for the future, because it will be here sooner than you think.

Most of us are already using many of these concepts in development. However, its always nice to have a good reference for supporting your opinions.

Further Reading

To learn more, read the book at http://www.faqs.org/docs/artu/.

You can also download a PDF version Download PDF.

Read More:

http://en.wikipedia.org/wiki/Unix_philosophy

http://www.linfo.org/unix_philosophy.html

Windows Live Writer

Today I am trying out Windows Live Writer. Its a pretty decent tool for publishing blogs. It can be downloaded from http://get.live.com/writer/overview.

However, when I attempted to install, an error message was shown saying that I needed Windows Update Agent  5.8.02469 or more. A little bit of googling revealed the following download link:

http://download.windowsupdate.com/…/7.0.6000.381/WindowsUpdateAgent30-x86.exe

Now I'm publishing my post with Live Writer. Soon I will also try out some of the free plugins.

I am also looking for a standalone installer for Windows Live Writer.

Pretty Logging...

Apache Chainsaw: A GUI Log Viewer

Introduction

Logging is an essential part of any software program. Logging provides a way to capture information about the operation of an application. Once captured, the information can be used for many purposes, but it is particularly useful for debugging, troubleshooting, and auditing. However, parsing a huge log file manually might be tedious, especially for server applications. Apache Chainsaw is a GUI based log viewer utility that makes things easier for us. To know more about Chainsaw visit http://logging.apache.org/chainsaw/index.html.

Motivation

Every once in a while we have to go through such scenarios. Often during internal development, customer testing and even in running products, log files are our only way of debugging any problem. Writing logs in a local file can be sufficient in single person development or testing. However, when we work in a team, distributed logging becomes necessary. The Apache log4j provides as options to send log messages to remote addresses, even broadcast them. Using tools like Chainsaw, we can observe the logs from a remote machine at real time. This can save a lot of time and effort.

Document Scope

In this document, I will describe the necessary steps for testing out Chainsaw with a simple logging program. The tutorial is divided into two parts. The first part shows how to configure log4j with SocketAppender. The latter part describes ZeroConf configuration.

Download Chainsaw v2 Bundle

Download chainsaw-bundle.zip from http://logging.apache.org/log4j/docs/webstart/chainsaw and extract it in CHAINSAW_HOME (In Apache Chainsaw’s homepage, another link is provided, but that didn’t work with ZeroConf).

Create Test Project

Create a new project ChainsawTest and add the following source file:

package ih.test.chainsaw;

import org.apache.log4j.Logger;

import org.apache.log4j.PropertyConfigurator;

public class ChainsawTest

{

// declare a static logger

private static final Logger log = Logger.getLogger(ChainsawTest.class);

public static void main(String[] args) throws InterruptedException

{

// load the log4j configuration file

PropertyConfigurator.configure("log4j_config.properties");

while(true)

{

log.debug("Testing DEBUG");

log.error("Testing ERROR");

// sleep for 1 second

Thread.sleep(1000);

}

}

}

Add log4j-1.2.8.jar to the project classpath.

Log4j Configuration (SocketAppender)

First, we will communicate with Chainsaw using a SocketAppender. The SocketAppender sends LoggingEvent objects to a remote a log server. In our case, Chainsaw will catch these objects and display in a GUI.

Add the following log4j_config.properties file:

log4j.rootCategory=DEBUG, zeroconf, chainsaw, stdout

# Socket Appender

log4j.appender.chainsaw=org.apache.log4j.net.SocketAppender

log4j.appender.chainsaw.remoteHost=localhost

log4j.appender.chainsaw.port=4445

log4j.appender.chainsaw.locationInfo=true

# Stdout Appender

log4j.appender.stdout=org.apache.log4j.ConsoleAppender

log4j.appender.stdout.layout=org.apache.log4j.PatternLayout

log4j.appender.stdout.layout.ConversionPattern=%d %p [%t] %C{1} - %mn

Here we have configured 2 appenders. For the SocketAppender we need to specify a remote host address and port. This is the address where logs will be sent. Since we will be running Chainsaw in the same pc, we use localhost. Port 4445 is default for SocketAppender (of course, any port number can be used).

We have also specified a Console Appender just for testing.

Running Chainsaw

Run chainsaw.bat from CHAINSAW_HOME. You should see the following window:

Select the second option (simple receiver on port 4445) and click OK.

View Logs

Run your project. The code will keep on writing two log messages. Now you can see that a new tab will appear beside “chainsaw-log” tab in the Chainsaw window. Open it and BINGO!!! There are your logs. Bright and shiny like the autumn morning.

For more information about how to filter and customize the UI, see the tutorial provided with Chainsaw.

Chainsaw with ZeroConf

One of the immediate disadvantages of the above SocketAppender is that we have to specify the address of the receiver of the logging events. If we want to have multiple receivers, we have to specify multiple appenders. Also we need to know the address of each of the receivers. Would not it be nice if we could just broadcast our logs to everyone listening? In that way everyone interested (testers, customers and other developers) could observe the logs using Chainsaw. ZeroConf is the solution.

To broadcast using ZeroConf, we need to use ZeroConfSocketHubAppender. It is a sub-class of SocketHubAppender that broadcasts its configuration via Zeroconf. This allows Zeroconf aware applications such as Chainsaw to be able to detect them, and automatically configure themselves to be able to connect to them.

To use ZeroConfSocketHubAppender copy jmdns.jar and log4j-zeroconf.jar from CHAINSAW_HOME and add them to the project classpath. Then add the following lines to log4j_configuration.properties:

# ZeroConf Socket Hub Appender

log4j.appender.zeroconf=org.apache.log4j.net.ZeroConfSocketHubAppender

log4j.appender.zeroconf.name=MyZeroConfSockeHubAppender

log4j.appender.zeroconf.port=4560

Once configured and your application re-started, you should be able to click on the Zeroconf tab inside Chainsaw, and see the "MyZeroConfSocketHubAppender" listed. If you double click on the row, Chainsaw will automatically connect to your application and start receiving events. You can tick the 'auto-connect' option to have Chainsaw immediately connect as soon as it sees your application started. Great for Dev/QA environment.

The Zeroconf-enabled SocketHubAppender broadcasts its existence via a multicast protocol, passing enough information for Chainsaw to be able to connect to it.

ZeroConf and Firewall

Multicast protocols generally don't pass through firewall, so in a production environment Zeroconf won't work.

Conclusion

As you can already see, Chainsaw can greatly simplify development and testing. No more ‘Dear Customer, please send us the 300MB log file and you will take a look’ or ‘Hey my log file shows ERROR, come take a look’. With a little bit of configuration we can sit on our chairs and watch logs from a remote machine in a nice GUI.

Thinking....

Thinking about what to put in my blog..... hmm....

My first blog

Today Nabila suggested that we should start writing blogs because everyone is doing it. So here I am.....