Eddy's occasional random thoughts

Like most engineers, technology is my comfort zone. It's easy to forget that running a successful business needs a sprinkling of sales & marketing, and even (gasp) self-promotion. So for a change, I thought I'd talk a little about Snoopdos itself and why you might want to consider us for your next project.

There are many reasons for choosing Snoopdos. Our team of experts combines highly communicative people with varied skills in a wide range of sectors which means we have a wealth of knowledge and creativity to offer you. We have collaborated internationally with great success. Our scope spans all kinds of projects for companies of all sizes. We can devise a program with which to achieve your goals. Take your business to the next level with our help!

We offer indepth consultancy services. Typically, we first evaluate your current set up. We do a general evaluation and quickly pinpoint what needs to change and how exactly to do so. We consider what positive ideas are already in place and design ways to make them even more powerful. There is no set program, since all companies differ, so we customize everything down to the details.

Whether you are an artistic start-up company or a big telecommunications operation, whether you have the most basic of sites or a large powerful one like O2 we can show you how to get more out of your online presence.

Don't underestimate the power of an effective website. An excellent website will not only provide great promotion and attract more clients to your services and products; done right, it can also open another stream of income for you as you select appropriate advertisements and links to display. Take your company international with the reach of an amazing site. We can get you there!

One of our core skills is software development. If your website or indeed other project requires bespoke development, we can deliver. Our team is extremely professional, able to work to deadlines and your specific requirements.

Please fell free to contact us at your earliest convenience so that we can schedule an initial meeting. We can discuss your needs and together decide on the best plan of action. We look forward to helping you accomplish your professional goals. Thank you for considering Snoopdos!

Yesterday, my Outlook 2003 hit the 2 GB limit on its PST file, and almost caused me to lose some important email.

To make me feel better, I'm describing the circumstances here. While I doubt anyone else affected will read this before suffering the same fate, they may at least gain some comfort after the fact from knowing they are not alone...

Anyway, the scenario goes like this. Outlook 2003 has a documented 2 GB limit on PST files (i.e. your Inbox and direct sub-folders). Unlike earlier versions of Outlook, which would happily corrupt your PST file with no warning if you exceeded the 2 GB limit, Outlook 2003 now traps this and (eventually) displays an error message saying the file is full.

There are two side effects that are not so obvious. The first is relatively benign: any messages in your outbox remain there after successful transmission, because there was no room to move them to Sent Items. Thus, you mistakenly think they haven't been sent for some reason and waste time trying to send them again; the recipient often ends up with multiple copies.

The more serious bug, and the one that really annoyed me, is that when Outlook downloads new messages, it realises it has nowhere to store them ... and just throws them away. My account is configured to leave messages on the mail server for two days after download, but I expect most users stay with the default account settings, which delete messages as soon as they are downloaded. Such users would likely not even be aware that they had lost mail, since it is irretrievably gone, with no record that it even existed. Certainly, Outlook gives no indication that anything is amiss.

Because this happened to my wife a couple of months ago, I knew to check my mail server's webmail interface for lost messages, and sure enough, there were several sitting there. When I freed up space on Outlook and downloaded my mail again, it happily ignored those messages, since it considered them already fetched, even though it had discarded them at the time.

Unfortunately, I forgot to check my secondary account's webmail interface as well, so only became aware of yet more missed messages when someone followed up to see why I hadn't responded.

Come on Microsoft, how hard would it have been to do this right??

(I'd upgrade to 2007, but I can't stand the new ribbon strip.)

My personal laptop is a Sony Vaio V505CP. It's almost five years old, but after upgrading it to 1.5 GB RAM and a 160 GB hard drive, it does a good job of running Windows Vista SP1. I use it mainly when I'm out on consulting jobs or travelling, so I don't need anything more powerful.

A month or two back, I noticed that whenever I played music, the audio would glitch every 20-30 seconds. The mouse pointer also froze briefly during the glitches. This was something new -- it was working fine after the original upgrade to Vista SP1. Today, I finally got around to investigating the cause.

For those in a hurry: the culprit was NDIS.SYS, the network driver. Simply turning off the built-in wireless adapter (easily done with a switch on the front of the Sony's case) made the glitch go away. I can live with this for now, since I rarely use wireless and stream music at the same time; normally, I leave the wireless enabled all the time, just in case I might need it, but it's not a big deal to turn it off.

So how did I figure this out? Here are the steps I followed, including dead ends (the journey is often as interesting as the destination).

My first port of call was Windows Task Manager. I usually start here, simply because Task Manager is standard on every Windows PC. In this case, it showed a big CPU spike whenever the glitch occurred, but unfortunately, Task Manager itself froze during the glitch so I couldn't identify what process was responsible.

The next tool to try was Process Explorer from SysInternals, a wonderful tool for all kinds of system probing. It includes a dummy task entry for DPC (Deferred Procedure Calls) and so I could see that during glitches, DPC was particularly active, responsible for almost all the additional CPU usage.

Deferred Procedure Calls are used whenever a process needs to make a system call, but the system is busy doing something important like handling an interrupt. The system call is queued until the interrupt completes, and all outstanding calls are then executed in sequence -- hence the brief jump in CPU activity.

Some Googling brought me to a handy tool called DPC Latency Checker, which graphs DPC usage over time. Using this confirmed the theory:

The red peaks indicate unusually long DPC latencies, which will cause system problems. Unfortunately, the checker didn't tell me the cause of these, but I knew it was probably a badly written device driver. The checker's website suggested using Microsoft's RATTv3 developer tool to identify the culprit.

I hadn't come across RATTv3 before, but it's very useful -- if you're running Windows XP, that is. It records the latency of each and every device driver's interrupt calls, correlated over time, and identifies the bad ones for you. Unfortunately, it doesn't work too well under Vista.

Back to Google, where I found a thread suggesting Microsoft's new Performance Toolkit as a good Vista alternative. This does everything RATTv3 can do, and a lot more besides.

One thing to watch out for: after installing the toolkit, all the files end up in "C:\Program Files\Microsoft Windows Performance Toolkit". I suggest copying them to C:\XPerf for convenience, since the installation folder is not added to the system path automatically.

After installing, you can begin a new capture by opening a command prompt in the toolkit folder and typing:

   xperf -on DiagEasy

to begin tracing. Then let the system run for a minute or two, until the glitch has occurred. Next, turn off tracing and convert the results to a file suitable for viewing:

   xperf -d trace.etl

Finally, view the accumulated results:

   xperf trace.etl

Pretty easy! Doing this produced a summary page with a ton of detail, most of which I didn't need (CPU usage, disk i/o, etc.) The thing I was interested in was DPC activity, which it showed as a graph like this:

The peaks in the graph show unexpectedly high DPC levels. By selecting one of those peaks and right-clicking, I was able to display a summary of all the driver activity that had contributed to the peak:

From this, it was clear that NDIS.SYS was the culprit, with a worst-case latency of more than 200 ms. (I repeated the test several times to confirm this, and it was consistently in top place.)

So mystery solved. From here, it was an easy step to try disabling the network adapters to see if that fixed things -- and turning off the wirless adapter did the trick.

So when I have a chance, I'll upgrade my wireless driver and hopefully that will sort it out permanently. For now, though, I'm just glad to have properly working audio again.

Marc Andreessen's blog today mentioned this cool demo:

This uses video goggles with a head-tracking sensor to remotely control the orientation of a camera mounted on a pilot-less plane, letting you virtually explore the heavens.

Apart from the general wow-factor of flying around the sky without ever leaving the ground, it reminded me of another piece of impressive technology I came across recently: quad-copters.

Here, a high-speed DSP is used to combine realtime feedback from gyros and sensors on position, wind direction, etc. to control four rotating blades independently allowing for stationary hovering in a wide range of conditions with no pilot input required. Great for remote video surveillance etc.

Combining these two pieces of technology seems like a perfect opportunity. Has anyone done it yet?

And a missing piece of the puzzle: even using stereo cameras to feed the video goggles, the image will still be flat since there is no way to remotely focus it (other than relying on auto-focus). Has anyone developed a set of video goggles that can track the eye's ability to focus on specific objects? Combine that with a pair of remote cameras that can track the eye's focus in that way and you could have REAL telepresence (once the latency isn't too high, of course).

Isn't it great that we live in an age where such amazing technology is affordable enough to let people devise interesting hacks in their spare time...?

Thanks to Kieran for pointing me towards this impressive Wii Remote hack, covered by Engadget here.

Johnny Chung Lee has done a marvellously simple hack which uses the standard Wii remote controller, plus some ballpoint pens modified to emit infra red, to convert any surface into an interactive touch-screen. With multiple pens, you can support multi-touch effects (as seen on the iPhone and iPod Touch), and previously mentioned on this blog back in March 2006.

Here's an example of his technique in use:

(Make sure you watch the video long enough to see the technique in action; it's very impressive, especially when combined with a video projector.)

Johnny's software to make all this work is free, and available here.