I said I'd never blog

The coding frenzy will certainly slow down.

My sabbatical vacation has come to an end. And to sum things up, I started this blog not because I got bored (even though I was afraid I would just end up geeking on nothing in particular all day long), but from a suggestion from fabienne: « why don't you document what you do, when you find solutions to your problems that google did not know about? ».

Very good suggestion indeed, and even though I could not document everything I did this month, I wrote a how-to and linked to a few interesting (or less interesting) pages. I'm glad that my friend oz found it interesting enough (or was simply really bored) to create an RSS feed of my stupid plain HTML page. The problem it creates for me now is that if I continue to add stuff to this page, it will become annoyingly long to load and still have no permanent links. I'm gonna think about that.

Anyway, all in all, even if (as always) I could not do as much as I wanted, I think I can be satisfied (maybe I'll even be proud one day) that my libc combined statically with my shell can now assist it enough to launch commands and eventually, be, hmm, somewhat, useful. Sadly, when used as the init program on NetBSD/i386 it crashes for some reason, and it still has more function stubs than actual code...

As mentioned earlier, getting it to work took more than expected, and was not exactly heavily documented. As too often, the answer is in the source, and kudos to free software for that, but it's often not really helpful (which is part of the reason why I am writing my own code in the first place). On the other hand, I really improved with gdb and reverse-engineering with objdump. Not that I dislike IDA, but it just forces one to really get the picture oneself, and the lack of practice over theory bit me again.

I should stop the chit chat now, and get to actual content, but I can't really sort out what I should have documented myself earlier (insert shame here). So, here it is, your moment of zen...

About developping a libc

Thankfully, smaller projects than glibc or BSD libc exist to get good inspiration from. These two were the first to help me figure that out:

• _start (the actual starting point in a program before main) should never be part of the shared object itself (libc.so), as it is meant to load it properly (among other things)
• this also means that if you don't load it properly, the afore-mentioned shared object will (almost) never work; as a consequence the libc supports only static linking to this date
• the compiler may expect you to restore the content of some registers to their original value, even after a function call (notably %ebx on i386) although I could not confirm this anywhere (and on Sparc I must say that it is actually part of the design and specification)
• some syscalls have weird undocumented particularities, take for example NetBSD/i386:
  • fork() in the source is considered to return the child's PID in both the parent *and* the child, where %edx (sic) actually allows the caller to make the difference
  • execve() will not return -errno in %eax but directly its actual value (thanks again guys)

As a conclusion, I have to say that Linux emulation on NetBSD was reliable to the point that everything crashed in the exact same way that my actual Linux host did. Quite amazing. I could perform both development efforts just on my NetBSD host, since my libc allows you to compile native Linux binaries using the plain standard gcc installation there (with CPPFLAGS="-U __NetBSD__ -D __linux__"). This was also part of my plan and works :)

Update: of course the third point was because of the cdecl calling convention.

From GCC manual:

       -fPIC
           If supported for the target machine, emit position-independent
           code, suitable for dynamic linking and avoiding any limit on the
           size of the global offset table.  This option makes a difference on
           the m68k, m88k, and the SPARC.

Oh really?

khorben@syn:~/temp$ uname -m
i386
khorben@syn:~/temp$ echo 'void func(void) { func(); }' > fpic.c
khorben@syn:~/temp$ gcc -c fpic.c 
khorben@syn:~/temp$ objdump -d fpic.o

fpic.o:     file format elf32-i386

Disassembly of section .text:

00000000 <func>:
   0:   55                      push   %ebp
   1:   89 e5                   mov    %esp,%ebp
   3:   e8 fc ff ff ff          call   4 <func+0x4>
   8:   c9                      leave  
   9:   c3                      ret
khorben@syn:~/temp$ gcc -fPIC -c fpic.c 
khorben@syn:~/temp$ objdump -d fpic.o

fpic.o:     file format elf32-i386

Disassembly of section .text:

00000000 <func>:
   0:   55                      push   %ebp
   1:   89 e5                   mov    %esp,%ebp
   3:   53                      push   %ebx
   4:   e8 00 00 00 00          call   9 <func+0x9>
   9:   5b                      pop    %ebx
   a:   81 c3 03 00 00 00       add    $0x3,%ebx
  10:   e8 fc ff ff ff          call   11 <func+0x11>
  15:   5b                      pop    %ebx
  16:   c9                      leave  
  17:   c3                      ret

Not exactly the same code, is it?

No comment.

Haha, I hear you saying « it was about time the coding frenzy came to an end » but you're wrong. Or not completely right. I did relax over the week-end in Rome, but I also operated a major surgery on DaPortal's code.

One more week to go. Yeeeah.

If like me you were looking for some input on how NetBSD decides to emulate a random platform instead of running natively, although you built the binary on NetBSD with the native tools, you are definitely willing to read this mail.

An example on NetBSD/sparc:

$ ktruss ./basename
 14030 ktruss   emul(netbsd)
 14030 ktruss   fktrace                            = 0
 14030 ktruss   fcntl(0x4, 0x3, 0)                 = 1, 3
 14030 ktruss   fcntl(0x4, 0x4, 0x1)               = 0, 4
 14030 basename emul(svr4)

and on NetBSD/i386:

$ ktruss ./basename
  4756 ktruss   emul(netbsd)
  4756 ktruss   fcntl(0x4, 0x3, 0)                 = 1
  4756 ktruss   fcntl(0x4, 0x4, 0x1)               = 0
Usage: basename string [suffix]
  4756 basename emul(linux)

(this is my own implementation of basename)

And again. Yeah. What is it about this time? Riiight.

Oh yes, this is about installing NetBSD on a diskless SGI Indigo2 machine.

Yes, I can post stuff before midnight, I'm awake enough. I just wanted to say I am happy that after a long and pointless discussion on IRC about it, some people actually understood my point about an error code path optimization in NetBSD libc's implementation of uname(). It's nothing really important (and really unlikely to be executed at all) but there was room for improvement, and I think this alone can justify it (but I'm obsessive compulsive).

I still don't understand why the utsname structure would need a single byte zero'd in case of an error (since it's, hmm, an error and no one should try to read the struct ever after), but at least I am not alone in this anymore :)

I just lost hours stupidly because of what is certainly a (security) bug in Apache 1.3.33/PHP 4 on Debian sarge (default configuration). For some reason when installing my PHP Explorer on a fresh install of Apache I could reveal the source code of one file (and only one) of the application.

I don't know how to reproduce it. Basically, what I did was just installing apache, php4 and libapache-mod-php4, and uncompressing the explorer in a "explorer" sub-directory. Then, pointing the browser to "http://host/explorer/explorer" (instead of "http://host/explorer/explorer.php") directly revealed the source code.

What's puzzling me even more is that I get an older version of the file that's actually there. Yeah, because when I saw that, I realized I made a typo in the comments. Ok, fixed it, hit refresh, still there. So, cleared cache, killed apache, restarted web browser, still there. Now, fucked what?!?

Working around it was pretty simple: disabling "MultiViews" for /, which is "On" by default. What I don't understand is why it reveals the source code for the PHP file, instead of choosing the ".css", ".js" or ".tpl" one and actually displaying it. Replicating this scheme for other files did not reveal anything.

If you can think of anything, please contact me. I won't report on this until I understand better.

Update: not quite the same, but still of some interest.

...not looking so good. It's not exactly what I was expecting (in spite of my global knowledge of the RISC platforms) and very different from the platforms I knew down to the assembly so far (which are only x86 and memories of m68k). I will collect here a series of links as I go.

• The reference: V8.pdf (400 pages)
• Nice introduction: sparcisa.ps.gz
• Something you need to know: the delay slot, also described here
• Another good introduction
• A focus on the stack
• A beginning of a reversing point of view
• A discussion about compiler output
• A beginning of a wikibook

\o/

Not so much to be proud of, but my first kernel patch was committed. w00t.

Finally!

It still does not do 4,2% of what I want it to do, but I can probably show this shamelessly enough:

$ ./UWfirmforce -vht 100 wg602v3_1_0_8.trx
Analyzing file: wg602v3_1_0_8.trx
Matching TRX signature #0 at offset 0
length 1634304, flags 0, version 1, offset #0 0x0000001c, offset #1 0x0008a24c,
offset #2 0x00000000
Score: 100%

Matching GZIP signature #0 at offset 1c
compression deflate, flags FNAME, 14/10/2005 02:54:20, OS Unix
Score: 100%

Matching CRAMFS signature #0 at offset 8a24c
size 1064960, flags VERSION_2|SORTED_DIRS, 768 blocks, 205 files, name
"Compressed"
Score: 100%

$ ./UWfirmforce -vt 100 mss122_openmss2.bin
Analyzing file: mss122_openmss2.bin
Matching BRCM signature #0 at offset 0
3 sections:
type tag, size 101
type flash, size 1617920
type disk, size 5877760
Score: 100%

Matching TRX signature #0 at offset 145
length 1617920, flags 0, version 1, offset #0 0x0000001c, offset #1 0x00119348,
offset #2 0x00000000
Score: 100%

Matching GZIP signature #0 at offset 173
compression deflate, flags FNAME, 29/05/2005 17:38:40, OS Unix
Score: 100%

Matching CRAMFS signature #0 at offset 1151961
size 65536, flags NONE, 317124106 blocks, 178190217 files, name "Compressed"
Score: 100%

Matching TRX signature #0 at offset 1618065
length 6250496, flags 0, version 1, offset #0 0x0000001c, offset #1 0x00000000,
offset #2 0x00000000
Score: 100%

Matching CRAMFS signature #0 at offset 1618093
size 6246400, flags VERSION_2|SORTED_DIRS, 5413 blocks, 2177 files, name
"Compressed"
Score: 100%

I don't think that everything is 100% correct, but I'll have a look.

By the way, the next release will be licensed under the GPL version 2.

I always wondered what the different colors meant in alsamixer, or more precisely, why the volume bars would just gray out. I thought it meant something like, "another application has polled/changed the volume since you last modified it through me". Having a quick glance at the source code, it is rather cryptic but has apparently nothing special about this.

It turns out ncurses apparently grays the bars if the terminal emulator tells it to refresh. Tell me if I'm wrong, but it's as stupid as that. There has to be a way to fix this (I seriously hope it's not a feature).

So, this week-end was pretty busy, with some fun and less fun times. To sum it up:

• fixed a bunch of stuff in my libc
• re-wired the servers while QSC also had some maintenance work (which partly explains the 8 hours of downtime this morning)
• helped fbz fetch hardware for her bed project
• patched NetBSD to support framebuffer in multi-processor mode
• another release of UWfirmforce, and a new plug-in for the strange BRCM header is ready

While I'm at it, there will be another downtime January 10th between 14h and 16h, because the electricity company also wants to re-wire stuff apparently.

I know you wanted to see this: my slippers!

I just added analysis callbacks for ROMFS and CRAMFS, which will make it in the next release of UWfirmforce. I got the hint from the openmss page that the TRX format is actually quite common over there, and it is indeed. I am now thinking about detection code for this format as well, and the firmware modification kit should be really helpful for this.

Update: I just released UWfirmforce 0.0.3, which includes this effort.

For some reason I started to think about the old LEGO Technic boxes that are left at my parents' place. This was a few months ago already, but these words must have slipped out of my mouth at some point as that's exactly what I got for last christmas. My first attempt at building a 4WD 4V monster buggy with independant suspensions unfortunately failed (and ended up as a helicopter, go figure), so I decided to try the lazy way.

That is, asking Daddy Internet for help. Mmh. I could not really find any freely-downloadable step-by-step book for anything, or at least nothing for my plain non-intelligent 90's LEGO blocks. I was expecting to find tons and tons of instructions for any kind of models my imagination would never have thought of, but only these two really caught my attention:

• MAN 8x4 Hook Lift Skip Tipper Truck
• Australian Off Roader

I would still be interested by more instructions. Maybe I just don't know about a special file format the community uses though.

Update: there is a file format indeed:

• LDraw CAD standard
• ldglite modeling program
• BrickSmith modeling program
• L3P interface to POV
• LEGO Set Database (I had 8024, 8055, 8064, 8074 and my brother had 8865, 8714)

It turns out this firmware looks very much like the one I showed during the demo. This should help me with the detection and decompression of files from ROMFS images for ÜberWall's UWfirmforce project.