about archive
Small Programs
February 23, 2010
 

If you want to produce small C++ programs with minimal dependencies using Visual Studio 2005 or newer then you're in for a world of hurt. Default setting after default setting is against you, and in the end you have to go completely off the reservation to get results. Business as usual in the Windows world. Previously I promised to talk about executable size without getting distracted by relevance, but I have a feeling that's not going to hold. I guess I'll just keep typing and see what happens.

Let's start with our friend from last time, HideAutoUpdate, and see what it takes to coerce Visual Studio 2005 into spitting out a 1.5kb exe. HideAutoUpdate makes for a good initial test case as it's barely more than a Hello World app, yet does something allegedly useful. As a refresher, here's its source code: 

// HideAutoUpdate by Chris Benshoof
//  the best flipping thing i'll ever write
#include <windows.h>
#include <tchar.h>

int WINAPI WinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance, LPSTR lpCmdLine, int nShowCmd)
{
    HWND dialog = FindWindow(WC_DIALOG, _T("Automatic Updates"));
    if (IsWindowVisible(dialog))
    {
        ShowWindow(dialog, SW_HIDE);
    }
    return 0;
}

That'll do, Pig. We're going to take this code and walk through each step to reduce the file size without sacrificing functionality or compatibility. In fact, just between you and me, we're going to accidentally increase the latter.

First we need a baseline. Create a new Visual C++ Win32 Project in Visual Studio, make sure to check "Empty Project" on the Application Wizard, then add a new C++ file and paste in the above code. Flip the solution configuration from Debug to Release. Now before we build, let's make this slightly less painful to follow along with at home. We can get Visual Studio to tell us the compiled file size in its output window which saves us having to go look for ourselves after each build, as that gets real old real quick. This can be done by having Visual Studio run "dir" on the program after every build. Such a feat can be accomplished with the project setting: 

Configuration Properties \ Build Events \ Post-Build Event \ Command Line
	cmd /c dir "$(TargetPath)"

Okay, now we can build and see that file size….

Release mode baseline: 6,565 bytes!

Oh snap, that's crazy small! 6.5kb, that works for me, especially since I didn't have to do anything! Ship it! But as the scroll bar to the right sadly foreshadows, there's more to the story. Believe it or not but that little HideAutoUpdate.exe we just compiled, that one that only calls three flipping functions, all of which have existed since 16 bit Windows aka The Time Before Christ… won't run on a single default installation of Windows XP, 2003, or anything earlier! Gadzooks!

We've incurred a dependency: Visual C++ projects default to link against Microsoft's Visual C++ RunTime dlls. Now wait, that's supposed to be a good thing! The C++ RunTime (CRT) contains the code for standard functions like printf(), operators like new and delete, and the behind-the-scenes work for building up and tearing down an environment for a C++ program to run in. It's efficient and sensible to have that shared code in a single set of dlls instead of baked into each and every exe, and it's even in the spirit of reducing a program's size which is what this article is about.

The problem is that Windows doesn't come with the CRT dlls that HideAutoUpdate needs. Back in the day, Visual Studio 6.0 linked programs against msvcrt.dll which came with practically every copy of Windows, even though that wasn't guaranteed, so a lot of programs got away without noticing this dependency let alone caring about it. That's not the case anymore. Each version of Visual Studio now has its own versions of CRT dlls. If built with Visual Studio 2005 then HideAutoUpdate requires msvcr80.dll which can only be installed through a 2.6mb Microsoft installer whose version must match at least what you had installed when you compiled. Requiring an installer over four hundred times the size of our little program is unattractive enough, but factor in the drama from Visual Studio service packs and automatic Windows updates rendering future builds incompatible with previous CRT dlls and… nah I don't think so I gotta boyfriend.

It seems I've strayed off track a bit since the topic is program size. To be fair though, dense stanzas of negativity are always on topic in Windows programming. Wait… come to think of it, I'm completely on track here, because when I talk about reducing program size what I'm really talking about is reducing dependencies. No… well, yes. Yes, I'm talking about reducing dependencies, but No, what I'm really really talking about is understanding dependencies, and that might just prove to be the most important thing ever. Yikes, a bold statement!

Now the wind is picking up. Let's cut to the chase. We drop our lame program's insignificant file size to 1.5kb by replacing Visual Studio's CRT with our own tiny custom one and tweaking a few linker settings. It's crazy hard and I'll elaborate on the how later. The file size is small, hooray, but there's something more interesting going on here: HideAutoUpdate doesn't depend on the Visual Studio CRT anymore. Now obviously that means the dll mess from above no longer applies but that can be accomplished simply by statically linking to the CRT, which causes the necessary portions of CRT code to be implanted directly into HideAutoUpdate at the cost of a much larger file size, but HideAutoUpdate avoids that too. And here's where independence gets its good rep: not depending on something means not depending on its dependencies. So what does the Visual Studio 2005 CRT depend on? A number of things, but of interest is the Windows API function IsDebuggerPresent(). This function simply returns TRUE or FALSE depending on if a debugger is attached to the program. I don't know why a call to it is in the Visual Studio 2005 CRT code, the why is irrelevant. It's in there, that's all that matters, and the function lives in kernel32.dll… starting with Windows 98. It doesn't get much more subtle than that. Out of the box, Visual Studio 2005 cannot produce a standalone C++ application that will run on Windows 95, because all C++ applications statically linked to the CRT depend on IsDebuggerPresent() which depends on Windows 98 or later. Windows 95 is unable to resolve that one link and will refuse to run such programs regardless of what they really do. HideAutoUpdate doesn't have this dependency and happily runs on Windows 95, which for all I know doesn't even have automatic update dialogs to hide.

It's pretty easy to imagine some responses to that. "Who the fuck cares about Windows 95?" is probably the chart topper. The answer is: "people who care", also known as "the right people". It's not about getting this program to run under Windows 95, it's about understanding why it won't, which means understanding its dependencies and their ramifications.

  • "Why does a program that only calls three functions require a 2.6mb installer?"
  • "Doesn't it seem like a program that only calls three functions should be less than 48kb?"
  • "Why can't a program that only calls three functions run on an operating system that has them?"

These questions are all about details. These details have nothing to do with what a program does, and their answers don't even need to make sense, but if nobody on a team is asking or caring about details just like these then I have concerns. Quality programs, regardless of function, feed on the socially devastating obsession to inconvenient, indirect, often otherwise inconsequential details that only nerds get right. Dependencies fall into that class of details. If a team of nerds can't get those right, what can they?

An example of this quality that comes to mind is µTorrent. µTorrent, the most popular BitTorrent client (outside of China), has kept itself a small standalone program since 2005 by using its own custom CRT code. The µTorrent team has even figured out how to get the best of both worlds: µTorrent manages to link to the old ever-present msvcrt.dll despite being compiled with Visual Studio 2008! That's extremely difficult and entirely unsupported, yet they go through all that trouble just to keep their program small without incurring the responsibility of managing dependencies. If µTorrent triples in size tomorrow and requires an installer, no one is going to stop using it. µTorrent's only function that matters is its ability to download interesting things. That its programmers care enough to take extreme measures to keep it lightweight fills me with a great confidence in that primary ability, a confidence that turns out to be well founded.

Another example is the Windows Task Manager. I was delighted to find out that Task Manager's original author avoided CRT reliance in the name of program size. Task Manager, the program that needs to work when all others don't, is very concerned about dependencies. I'd never thought to appreciate Task Manager, but reading Dave's account of the work he put in to make the program meet his self-imposed criteria for quality makes two things clear: Task Manager was built with love, and you want Dave on your team. Dave took the time to write flicker prevention code for his Performance tab because he cared, and his successors didn't even care enough to invoke that code on their Networking tab. What does that say about the code that actually manages tasks? I can only speculate, but I'm certainly glad Dave got to Task Manager first before they did.

Alright, then. Enough of that, back to this. Let's turn off dynamically linking to the CRT in favor of static linking. 

Configuration Properties \ C/C++ \ Code Generation \ Runtime Library
	Multi-Threaded DLL (/MD) => Multi-Threaded (/MT)

Rebuilding nets us… 49,152 bytes! All profit! Wait…

Well we ditched our dll dependency but now we're huge. Let's try going cold turkey on Visual Studio's CRT completely. 

Configuration Properties \ Linker \ Input \ Ignore All Default Libaries
	No => Yes (/NODEFAULTLIB)

Rebuilding gets us… 2 unresolved external symbol linker errors!

Well that's cold turkey for you. As I alluded to / blatantly stated earlier, we're going to use a tiny custom CRT of our own to get by. This is what I meant by going off the reservation. It's called minicrt and for now I'm going to completely gloss over the origin of this methadone and also kick these lame addiction metaphors. Kick 'em like smoking. Dammit! 

Configuration Properties \ Linker \ Input \ Additional Dependencies
	minicrt.lib

Rebuilding gets us… 3,072 bytes! And some warnings about section merging. Well, what doesn't these days?

In one shot we dropped a Visual Studio 2005 standalone program to a sixteenth of its size, and for what it's worth killed off the IsDebuggerPresent() dependency. That… seems like a big deal. In fact, that was the whole ballgame! Any size reduction beyond that is penny ante and just for fun. I know, right, what are we waiting for?

At this point HideAutoUpdate.exe has four sections, each of which is aligned on a 512 byte boundary, which is a fancy way of saying that they're multiples of 512, which to us means that each is a minimum of 512 bytes. If we can eliminate or merge some sections then the program gets smaller. First on the chopping block is the resource section .rsrc. The only reason a resource section is being built is so that the XML manifest that Visual Studio generates can be embedded there. The only use this interfaceless app had for a manifest was to contain the version info for the necessary CRT dll. Since we've eliminated that need, disabling the manifest will in turn eliminate the resource section. 

Configuration Properties \ Linker \ Manifest File \ Generate Manifest
	Yes => No

Rebuilding gets us… 2,560 bytes!

As prophesied, there goes 512 bytes. We're left with three sections: .text, .rdata, and .data. We're stuck with their contents so we'll merge them together. First up, merging .text into .data, and while we're at it let's silence the merge warnings. 

Configuration Properties \ Linker \ Command Line \ Additional Options
	/merge:.text=.data /ignore:4254

Rebuilding gets us… 2,048 bytes! And no more warnings!

That leaves .rdata and .data. Once more, with feeling: 

Configuration Properties \ Linker \ Command Line \ Additional Options
	/merge:.text=.data /merge:.rdata=.data /ignore:4254

Rebuilding gets us… 1,536 bytes! Ding ding ding!

It's that simple, and yet that hard. Or rather I just made it look hard with all the gabbing. Maybe a quick recap can save this:

Release baseline 6.5kb Depends on a specific version of msvcrt80.dll.
/MD => /MT 48.0kb Dll dependency elminated, but now we're huge.
/NODEFAULTLIB 0kb Visual Studio CRT eliminated but doesn't build. Well, you can't beat zero bytes.
minicrt.lib 3.0kb Crazy small! IsDebuggerPresent() dependency removed.
/MANIFEST:NO 2.5kb Eliminating resource section drops 512 bytes.
/merge:.text=.data 2.0kb Merging sections drops 512 bytes.
/merge:.rdata=.data 1.5kb Merging sections drops 512 bytes.

Numbers don't lie, replacing the Visual Studio CRT is how you make small programs small. Linker settings are peanuts. As we'll see next time, this is old news.

I'd be remiss if I didn't mention a few other settings. Though in this instance they didn't affect file size, they do under other circumstances or are at least in the same spirit. 

Configuration Properties \ C/C++ \ Optimization \ Optimization
	Maximize Speed (/O2) => Minimize Size (/O1)

HideAutoUpdate does so little that there's nothing this setting can optimize further, and so it has no effect on it. On non-trivial programs it makes a difference, sometimes significantly. Both settings, /O1 and /O2, expand into a set of individual optimization settings, most of which the two share so it's not an either/or thing. You get a lot of the same optimizations either way. 

Configuration Properties \ Linker \ Optimization \ Optimize for Windows98
	Default => No (/OPT:NOWIN98)

When set to Default, this optimization automatically kicks in once a program reaches a certain size. This optimization changes the section alignment from 512 to 4,096, which makes a program load quicker on Windows 98. For small programs that means each section costs a minimum of 4kb instead of 0.5kb. HideAutoUpdate is so small that Visual Studio didn't enable the optimization, but it should still be nipped in the bud. This setting was tripped when we first statically linked to the CRT and it contributed to the large 48kb file size. It would have been 37kb if the optimization was explicitly disabled. 

Configuration Properties \ Linker \ Debugging \ Generate Debug Info
	Yes (/DEBUG) => No

It seems questionable to me that this is enabled by default for Release configurations. This setting embeds a string in the program to the full path of the program database file on your computer, which could easily be something along the lines of, oh I dunno, "C:\Documents And Settings\D.B.Cooper\My Documents\Visual Studio 2005\Projects\HideAutoUpdate\Release\HideAutoUpdate.pdb". That feels like an inappropriate amount of information to be exposing by default, and frankly, a pretty lame way for me to go down. 

Configuration Properties \ General \ Character Set
	Use Unicode Character Set => Not Set

To set the character set to ANSI, set the character set setting to Not Set. Think about that the next time you wonder what's keeping normal people out of the industry. HideAutoUpdate has to use the ANSI versions of Windows functions to run on Windows 95, 98 and ME because ANSI versions are all those operating systems have. Otherwise I'd be in trouble for linking to FindWindowW() instead of FindWindowA(). Dependencies!

Let's never write this article again, shall we? Next time I swear to God I'm not going to mention a single compiler setting. Instead we'll learn about the origin of minicrt.lib. Well… I'll learn how to write an origin of minicrt.lib and you'll learn how to fact check it. Also, source code. Until then, enjoy the compiled library. For a lot of modest programs simply using this will shrink them without complications. Others… not so much. Eventually we'll see what can be done about those too.

minicrt.lib