Yesterday, my Outlook 2003 hit the 2 GB limit on its PST file, and almost caused me to lose some important email.
To make me feel better, I'm describing the circumstances here. While I doubt anyone else affected will read this before suffering the same fate, they may at least gain some comfort after the fact from knowing they are not alone...
Anyway, the scenario goes like this. Outlook 2003 has a documented 2 GB limit on PST files (i.e. your Inbox and direct sub-folders). Unlike earlier versions of Outlook, which would happily corrupt your PST file with no warning if you exceeded the 2 GB limit, Outlook 2003 now traps this and (eventually) displays an error message saying the file is full.
There are two side effects that are not so obvious. The first is relatively benign: any messages in your outbox remain there after successful transmission, because there was no room to move them to Sent Items. Thus, you mistakenly think they haven't been sent for some reason and waste time trying to send them again; the recipient often ends up with multiple copies.
The more serious bug, and the one that really annoyed me, is that when Outlook downloads new messages, it realises it has nowhere to store them ... and just throws them away. My account is configured to leave messages on the mail server for two days after download, but I expect most users stay with the default account settings, which delete messages as soon as they are downloaded. Such users would likely not even be aware that they had lost mail, since it is irretrievably gone, with no record that it even existed. Certainly, Outlook gives no indication that anything is amiss.
Because this happened to my wife a couple of months ago, I knew to check my mail server's webmail interface for lost messages, and sure enough, there were several sitting there. When I freed up space on Outlook and downloaded my mail again, it happily ignored those messages, since it considered them already fetched, even though it had discarded them at the time.
Unfortunately, I forgot to check my secondary account's webmail interface as well, so only became aware of yet more missed messages when someone followed up to see why I hadn't responded.
Come on Microsoft, how hard would it have been to do this right??
(I'd upgrade to 2007, but I can't stand the new ribbon strip.)
My personal laptop is a Sony Vaio V505CP. It's almost five years old, but after upgrading it to 1.5 GB RAM and a 160 GB hard drive, it does a good job of running Windows Vista SP1. I use it mainly when I'm out on consulting jobs or travelling, so I don't need anything more powerful.
A month or two back, I noticed that whenever I played music, the audio would glitch every 20-30 seconds. The mouse pointer also froze briefly during the glitches. This was something new -- it was working fine after the original upgrade to Vista SP1. Today, I finally got around to investigating the cause.
For those in a hurry: the culprit was NDIS.SYS, the network driver. Simply turning off the built-in wireless adapter (easily done with a switch on the front of the Sony's case) made the glitch go away. I can live with this for now, since I rarely use wireless and stream music at the same time; normally, I leave the wireless enabled all the time, just in case I might need it, but it's not a big deal to turn it off.
So how did I figure this out? Here are the steps I followed, including dead ends (the journey is often as interesting as the destination).
My first port of call was Windows Task Manager. I usually start here, simply because Task Manager is standard on every Windows PC. In this case, it showed a big CPU spike whenever the glitch occurred, but unfortunately, Task Manager itself froze during the glitch so I couldn't identify what process was responsible.
The next tool to try was Process Explorer from SysInternals, a wonderful tool for all kinds of system probing. It includes a dummy task entry for DPC (Deferred Procedure Calls) and so I could see that during glitches, DPC was particularly active, responsible for almost all the additional CPU usage.
Deferred Procedure Calls are used whenever a process needs to make a system call, but the system is busy doing something important like handling an interrupt. The system call is queued until the interrupt completes, and all outstanding calls are then executed in sequence -- hence the brief jump in CPU activity.
Some Googling brought me to a handy tool called DPC Latency Checker, which graphs DPC usage over time. Using this confirmed the theory:

The red peaks indicate unusually long DPC latencies, which will cause system problems. Unfortunately, the checker didn't tell me the cause of these, but I knew it was probably a badly written device driver. The checker's website suggested using Microsoft's RATTv3 developer tool to identify the culprit.
I hadn't come across RATTv3 before, but it's very useful -- if you're running Windows XP, that is. It records the latency of each and every device driver's interrupt calls, correlated over time, and identifies the bad ones for you. Unfortunately, it doesn't work too well under Vista.
Back to Google, where I found a thread suggesting Microsoft's new Performance Toolkit as a good Vista alternative. This does everything RATTv3 can do, and a lot more besides.
One thing to watch out for: after installing the toolkit, all the files end up in "C:\Program Files\Microsoft Windows Performance Toolkit". I suggest copying them to C:\XPerf for convenience, since the installation folder is not added to the system path automatically.
After installing, you can begin a new capture by opening a command prompt in the toolkit folder and typing:
xperf -on DiagEasy
to begin tracing. Then let the system run for a minute or two, until the glitch has occurred. Next, turn off tracing and convert the results to a file suitable for viewing:
xperf -d trace.etl
Finally, view the accumulated results:
xperf trace.etl
Pretty easy! Doing this produced a summary page with a ton of detail, most of which I didn't need (CPU usage, disk i/o, etc.) The thing I was interested in was DPC activity, which it showed as a graph like this:

The peaks in the graph show unexpectedly high DPC levels. By selecting one of those peaks and right-clicking, I was able to display a summary of all the driver activity that had contributed to the peak:

From this, it was clear that NDIS.SYS was the culprit, with a worst-case latency of more than 200 ms. (I repeated the test several times to confirm this, and it was consistently in top place.)
So mystery solved. From here, it was an easy step to try disabling the network adapters to see if that fixed things -- and turning off the wirless adapter did the trick.
So when I have a chance, I'll upgrade my wireless driver and hopefully that will sort it out permanently. For now, though, I'm just glad to have properly working audio again.
Marc Andreessen's blog today mentioned this cool demo:
This uses video goggles with a head-tracking sensor to remotely control the orientation of a camera mounted on a pilot-less plane, letting you virtually explore the heavens.
Apart from the general wow-factor of flying around the sky without ever leaving the ground, it reminded me of another piece of impressive technology I came across recently: quad-copters.
Here, a high-speed DSP is used to combine realtime feedback from gyros and sensors on position, wind direction, etc. to control four rotating blades independently allowing for stationary hovering in a wide range of conditions with no pilot input required. Great for remote video surveillance etc.
Combining these two pieces of technology seems like a perfect opportunity. Has anyone done it yet?
And a missing piece of the puzzle: even using stereo cameras to feed the video goggles, the image will still be flat since there is no way to remotely focus it (other than relying on auto-focus). Has anyone developed a set of video goggles that can track the eye's ability to focus on specific objects? Combine that with a pair of remote cameras that can track the eye's focus in that way and you could have REAL telepresence (once the latency isn't too high, of course).
Isn't it great that we live in an age where such amazing technology is affordable enough to let people devise interesting hacks in their spare time...?
Thanks to Kieran for pointing me towards this impressive Wii Remote hack, covered by Engadget here.
Johnny Chung Lee has done a marvellously simple hack which uses the standard Wii remote controller, plus some ballpoint pens modified to emit infra red, to convert any surface into an interactive touch-screen. With multiple pens, you can support multi-touch effects (as seen on the iPhone and iPod Touch), and previously mentioned on this blog back in March 2006.
Here's an example of his technique in use:
(Make sure you watch the video long enough to see the technique in action; it's very impressive, especially when combined with a video projector.)
Johnny's software to make all this work is free, and available here.
Marc Andreessen's always-interesting blog today talks about OpenSocial, a new standard spear-headed by Google which aims to provide a common API for embedding new web content and apps across all the main social network sites.
This is very similar to the Facebook Platform API launched a few months ago to critical acclaim, but with the significant difference that OpenSocial is an open standard which pretty much everyone else except Facebook is jumping on the bandwagon to support.
Marc's blog does a much better job of describing the benefits than I can. In a nutshell, though, it means that if you run a website that offers a useful service, you can now easily allow it to be embedded in any of the main social networking sites (LinkedIn, Friendster, Ning, etc.) by just adding some basic HTML & Javascript support.
This should be fun...
(The official Google launch is tomorrow, at which point http://code.google.com/apis/opensocial should become operational.)
:: Next Page >>
An occasional blog recording useful URLs, observations or other random thoughts I may have from time to time...
| Next >
| Mon | Tue | Wed | Thu | Fri | Sat | Sun |
|---|---|---|---|---|---|---|
| << < | > >> | |||||
| 1 | ||||||
| 2 | 3 | 4 | 5 | 6 | 7 | 8 |
| 9 | 10 | 11 | 12 | 13 | 14 | 15 |
| 16 | 17 | 18 | 19 | 20 | 21 | 22 |
| 23 | 24 | 25 | 26 | 27 | 28 | 29 |
| 30 | ||||||
Network World just pointed me towards an intriguing Wooden Periodic Table of the Elements.
It's really very nicely done - check it out.
I ordered a few bits and pieces from Big Pockets last week -- a bunch of blank DVD cases, some blank printable Ritek DVD-Rs, and a set of replacement colour inkjet cartridges.
They arrived at the office on Wednesday, packaged in a large cardboard box which originally housed a B&Q white toilet bowl. I can say this with confidence, because large pictures of the toilet were printed on the side of the box.
Since I was out of the office Wednesday & Thursday, everyone else had two days to ponder what I might have been up to that required urgent replacement of a toilet bowl...
Nice to see Big Pockets are recyling where possible!
In my continuing search for ways to improve traffic flow (rather ironic, since I'm working from home these days), I came across this analysis of how small variations in departure time for the morning commute could lead to significant time savings.
Friends who commute from North Wicklow to Dublin city centre had told me that there was a narrow window in the morning when you could get a clean run into the city without too much delay - about 10 minutes after the early morning rush has subsided but before the school runs kick in. This article appears to support that theory (albeit on a different continent!)
(Of course, there are the usual Slashdot comments on the article also...)
Maybe one day, I'll actually get to put all this theory to use...
You have to hand it to Microsoft - when they localise something, they don't do it by halves. I just came across a pointer to the Welsh version of Windows XP.
The big news this week was that Smart Telecom (the largest independent Irish telecomms company) announced a new residential DSL product with some very attractive features:
As usual, there has been plenty of discussion at boards.ie about this. Most people are impressed by the high bandwidth / low cost ratio, but concerned about the miserly 128 Kb/s upstream bandwidth.
Still, at €35, I expect they'll make a killing. It will be interesting to see if and how the other ISPs beef up their offerings to make them more competitive.
People continue to find new ways to explore Google Map's public API. Today, I came across this site:
http://www.sueandpaul.com/gmapPedometer/
It lets you measure the walking (or cycling) distance between any two points in the city, simply by double-clicking all the significant direction changes along the router. Simple, but effective.
New Scientist has an interesting article about a new robot with a knack for finding its feet.
While there have previously been some impressive demonstration videos of robots from Toyota, Honda and Sony, this is the first robot I've seen that actually mimicked human like movement patterns.
You can download the 13 MB demo video here; it's rather unsettling to watch.
I just hope they have a good team working on the software...
I’ve just returned from the Consumer Electronics Show, which ran from January 8-11 in Las Vegas. This was my first time attending the largest trade show in America (2700 exhibitors, 150,000 attendees) and it was quite an experience. Over the next few days, I'll post entries about some of the more interesting products and companies I saw.
I was travelling with some colleagues, and we had four days to cover the exhibition – just as well, since it covers ten large convention halls, and conference space in several nearby hotels.
The scale of CES is hard to grasp – by lunchtime on the first day, we had cleared half of one hall and felt we were doing well – until we realised we had only cleared half of one corner of one hall!
It was time for a more aggressive approach. By planning ahead, and stepping up the pace substantially, we managed to cover about 90% of the exhibits during the first three days! This left Thursday morning for some final follow-up visits to stands, and we were all done by lunchtime, leaving the afternoon for some R&R.
The conference was well organised, with courtesy coaches between the different venues, free Internet access available in the halls (supplied by DivX), and even free international phone calls courtesy of Vonage, who had installed VoIP kiosks in the lobbies. Complimentary magazines from just about every consumer and electronics publisher you've ever heard of were also available outside the conference entrance, along with the six show guides and product directories.
While most of the halls were in the gargantuan Las Vegas Convention Center, one hall was in the Sands Expo Centre, about a mile away. (The Adult Entertainment Expo was taking place in an adjacent hall during the week, which led to an interesting mix of attendees in the areas outside the hall.)
Two products were in evidence just about everywhere this year – large high-definition LCD televisions, and digital photo frames. Even manufacturers who didn't sell photo frames were using them to display information sheets for their actual products.
There was a wide and varied selection of Windows Media Centre PC cases from assorted vendors – perhaps the most impressive (if not the most practical) were the Diamond Media Centre range from Moneual, which featured a jewel-encrusted fascia panel and control knobs at prices ranging from $30,000 to $1,000,000, they are for elite purchasers only.

While most of the larger vendors had extravagant stands with large floor areas, there were plenty of smaller stands there as well. The entry level cost for a stand is in the region of $6000-$8000 depending on location (about 2 metres x 3 metres) which is within reach for even small startups.
There were surprisingly few freebies on offer. Other than the ubiquitous carrier bags, and a few trinkets like LCD pocket torches, watch batteries, and T-shirts, pickings were slim.
One of the biggest announcements didn’t even happen at the show. Apple were running MacWorld in San Francisco during the same week, and launched their new and long-awaited iPhone, though it won't be available until June. While the industry was impressed with the product, there was plenty of scepticism about Apple's ability to break into the highly competitive cellphone market, especially with a product that costs $600! Time will tell...
Alongside the main CES convention, there were numerous seminars and panel sessions to discuss issues of interest to the consumer electronics industry. Most of these cost money to attend, but some were open to all CES participants.
The first session we attended (on Monday afternoon) was CNET's Next Big Thing, a discussion about three hot areas of debate in the industry at the moment:
Several industry panelists (including a CNET reader flown in from the UK to represent the consumer) discussed these items at length; the audience then voted electronically on how they thought things would develop in the future.

On mobile video, the verdict was: yes, there is a new and growing market for media playback on mobile devices, but access needs to become far less restrictive (both in price, and range of media available) before it will take off. Mobile video will supplement existing markets rather than replacing them.
Dynamic content like news clips, music videos, and short length TV programs are more suited to these devices than feature length movies. One panelist predicted that the production style of programs will change to match the constraints of the mobile devices – more talking heads, fewer panoramic landscapes, etc.
All reasonable points. However, the most important point was only briefly mentioned: mobile video need not be streamed. People use it to fill time when they would otherwise be bored (standing in a queue, waiting for something to complete), and it suffices to have pre-loaded content available on their mobile device; it's not a big deal if it's several days old.
With modern video codecs and ever-increasing flash capacities, it's quite feasible to store multiple movies or (more likely) TV programs on your portable device. I think this trend is just beginning to gather stream.
The panel were (eventually) united in their view that DVD wouldn’t be going away anytime soon – certainly not by 2010, as had been suggested. The tangible nature of a DVD, which makes it collectable, suitable for gifts, easy to use and distribute, etc. all combine to make it unlikely to disappear.
However, the panel did did agree that online streaming media will gain significant following as well. (No surprises there.)
I'm broadly in agreement with this view. It takes a long time for a given technology to die out; after all, there are still plenty of VHS tapes around, even though it is now difficult to buy a VHS recorder.
I also doubt we'll see broad acceptance of streamed video until most households have a fast broadband connection (where 'fast' means 10 MBps or higher, with no contention).
Digital Rights Management (DRM) was the most contenious discussion, with representation from the legal profession, movie industries, and consumers, as well as several industry observers. Everyone agreed that some sort of rights management was essential, but making it transparent enough so that consumers were not unduly restricted when they tried to use media they believed they owned was judged difficult.
Apple's ITunes was given as an example of how to make DRM straightforward, so that customers are not always even aware that any special restrictions apply.
The biggest hinderance to existing DRM technology is that it goes against the grain of what consumers are used to. If you buy a CD, you can take it to a friend's house to share with them or loan to them. You can play the same CD in your living room, your bedroom, your car, on your laptop, and anywhere else that has a CD player.
This is almost never true for media protected with DRM, especially where that media has been downloaded rather than delivered in some sort of physical packaging. While providers are entitled to protect their content, they need to find a less onerous way to do so than has currently been attempted.
I have no doubt that this problem will be solved; until then though, consumers will continue to be highly suspicious of anything resembling DRM.
CNTs are Carbon Nanotubes, a new technology that allows flatpanel screens to be constructed from what is essentially an array of thousands of tiny CRTs, each CRT representing a pixel or so.
A friend (thanks Mike) pointed me towards this company which has been experimenting in the area. Click on Demonstrations on the left to see a nifty little animated demo of their current prototype (and be sure to read the caveats as well as to why the quality isn't wonderful yet).
Why is this stuff of interest? Primarily because the potential exists to make absolutely massive screens using this technology. Plasma and LCD screens are limited in size, because they rely on a single sheet of semiconductors to make the whole screen - the technology to seamlessly stitch together multiple panels to make a bigger screen doesn't exist.
For those screen types, cost increases rapidly as the size gets larger, because it becomes more difficult to get screens with an acceptably high number of working pixels.
With CNTs, this is no longer an issue - you simply manufacture buckets of tiny screens (say 4" x 5") and stitch them together to make as big a display as you like. Tiny screens means much higher yield which in turn translates to lower prices. Plus, as a bonus, you get all the benefits of a normal CRT (bright picture, fast refresh, etc.) without the drawbacks of massive weight and depth.
Probably another year or two to go before these hit your local electronics superstore, but it can only be a matter of time.
(On the other hand, whatever happened to bubble memory?)
Eddy
A friend referred me to this website today:
http://www.mackers.com/projects/dartmaps/
It's a very cool Google Maps hack that scrapes realtime DART position info from Irish Rail's website and plots it on a Google Map of Dublin.
Be sure to zoom in - you can actually see the trains moving in realtime. Click on a train to find out its destination.
Amazing what you can do with a bit of XML, scripting and some ingenuity...
For those of you who use it, my DVD list now sports a handy search field ... enjoy.
Time for yet another mapping link:
SketchUp is a rather cool plug-in for Google Earth that allows you to add real 3D buildings to the Google Earth city view.
Check it out...
I hadn't been paying much attention to this AJAX thing, until I saw an article about it in Dr Dobbs Journal (Feb 2006).
AJAX is Asynchronous Javascript And XML, essentially a collective name for a group of technologies that together allow web pages to asynchronously interact with a remote server without losing the page context. This leads to the ability to design significantly more interactive interfaces.
The best example I've seen is the one cited in the Dr Dobbs article: Google Suggests. As you type in search keywords, a realtime display appears below showing you suggested matches for what you're typing. All the words retrieved are fetched from Google's server in realtime as you type, without you even noticing. Very cool...
Steve pointed me towards an interesting demo video showing off the capabilities of the HAL 2000 home automation system. It lets you control your home using speech commands from any room - quite impressive.
The author has an extensive discussion of the technology involved (speakers, distribution amps, microphones, etc.) along with some comments on the limitations of the system.
Startrek's onboard computer suddenly doesn't seem quite so far away...
Shortly before Christmas, one of my media drives (a 300 GB Maxtor) started to give warning signs that it was about to die -- a number of files mysteriously developed CRC errors for no apparent reason.
I downloaded Maxtor's PowerMax utility which gives the drive a once-over. PowerMax can run a 90 second test to give a quick estimate of the drive's condition, or a more thorough test that lasts several hours. In the event, it took only 10 seconds to figure out my drive was quite sick and should be sent back to Maxtor for replacement. (The drive was just over 12 months old, but to my relief, it turned out to have a three year warranty. Exactly the same drive sold in the USA comes with only a one year warranty. Go figure.)
I bought a replacement 300 GB drive to back up the data (luckily, I only lost a few files), and then kicked off Maxtor's RMA process. Conveniently, if you give them your credit card number, they will ship you the replacement drive first -- you can then use that drive's Maxtor-approved packaging to return the defective unit.
Three weeks later, Maxtor Support emailed me to say that it would be February before an exact replacement for my unit would be available, but that they'd be happy to ship me one of their new DiamondMax 10 6B300R0 models instead if that was acceptable. While RAID users might have preferred an exact replacement, I just wanted a new drive (and this one was better specced) so I told them to go ahead.
The new drive duly arrived about two weeks ago, and I decided to install it as the boot drive in my main PC; I partitioned it as a 30 GB C drive for Windows with the remaining 250 GB for my D: drive. (Yep, that's only 280 GB, but that works out as 300,647,710,720 bytes -- got to hand it to those hard drive marketing guys...)
After installation, it slowly became apparent that something wasn't quite right with my system. Simple file copies from one partition to another (and especially from firewire drives to my D: drive) seemed much slower than they should be. Worse, while this was going on, Windows would slow to a crawl - in some cases, I could see individual window elements rendering a few pixels at a time.
It took me a while to figure out the drive was the issue - for normal use, things worked fast enough. I checked the DMA settings on the drive (Windows XP has a tendency to disable DMA without telling you, if it detects errors) but it was fine. Eventually though, I noticed that HDTach 3 showed rather erratic performance - instead of the usual smooth graph, it was very choppy.
That got me thinking, and I finally decided that the culprit might be the removable drive caddies that I use in all my systems. These are only rated to ATA66, yet the new Maxtor is an ATA133 drive. A few minutes later, I'd whipped out the caddy and connected the drive directly and -- everything back to normal! I also replace my ATA66 cable with a newer ATA133 cable, which improved the HDTach results by another 10% or so.
I can only presume Windows must have been retrying like crazy before, hence the heavy CPU usage -- or possibly switching briefly to PIO mode to recover from a corrupt DMA transfer? In any case, it's good to have a system at 100% again!
Lesson learnt: adding any new hardware to a working system should always be done very carefully!
Seagate had a cute visual exhibit at CES 2007 - a typical laptop hard disk, represented as an equivalent number of CDs (assuming you only store MP3s, of course.)

Seagate's point, of course, is that you might want to think about backing up that hard disk ... and naturally, they had a product designed to do exactly that.
2004 was definitely the year of the iPod. I bought one, my brother bought one, my sister brought one, and even my Dad bought one.
Unfortunately, though, iPods sold in Europe have their maximum volume capped compared to similar US models - in fact, the maximum level is about 30% lower on the EU models.
This is apparently to comply with EU legislation. Unfortunately, because it affects line-level output as well as headphone output, it causes all sorts of problems when you use something like an iTrip or connect the iPod to your hi-fi or car radio's line-in - the iPod volume is significantly lower than that of the other equipment, so you spend a lot of time adjusting volume as you switch between your iPod and radio or CD.
There have been several hacks that attempt to overcome this by tweaking a volume compensation tag in each MP3 file you download to your iPod. However, these are not ideal - you have to remember to do it whenever you update your play list.
I finally found a utility that did what I really wanted: just remove the cap entirely. The utility is goPod and it simply adjusts a field in the firmware to indicate that capping is no longer required.
It works really well. If you have an iPod in Europe, you need this.
From the Useless But Cool dept: Andrew Carol (no relation) has built a Lego Difference Engine in the fashion of Charles Babbage's original.
Due to structural limitations of Lego (plastic bricks and gears can't handle the same stresses as metal), it's a cut down version but still functional. Very cool.
And on the subject of Lego, if you've been playing with Lego Mindstorms (Lego's robotic kits), Lego have just announced Mindstorms NXT, the next generation - with a 32-bit RISC processor and a number of other significant improvements. My second childhood should be due any day now, and I'm looking forward to playing with this...
I came across this article about RAID systems under Linux, and some of the problems that can occur.
It's interesting reading, especially the explanation as to why a single drive failure in a Linux RAID system is often also followed by a second failure before the array has had a chance to rebuild ... rather concerning, since that can effectively negate the redundancy benefit of having a RAID in the first place.
Let's hope my MegaRaid SATA150-6 controller doesn't suffer from similar problems...
Also of interest: drive manufacturers appear to be routinely making minor adjustments to the size of discs, even when the model number remains the same - this can cause problems when swapping out a dead drive, if the replacement is slightly smaller. Might be an idea to stockpile a few spare drives to have on hand which are from the same batch as your original set.