Winfred C. H. Lu

GDB Debugging ARM U-Boot on QEMU

2011-12-29T02:44:00.000-08:00

1 Introduction

QEMU is a generic and open source machine emulator and virtualizer. In this article, we will use it as a machine emulator and run the U-Boot on top of it. It will also be demonstrated about how to debug U-Boot with GDB.

2 Installation and Compilation

2.1 Cross Toolchain

Install one of the cross toolchain accordingly. available toolchains:

2.2 QEMU package

Install Debian QEMU package.

# apt-get install qemu

Or on a host running Ubuntu, try

$ sudo apt-get install qemu-kvm-extras

We may also check if ARM versatilePB is supported by QEMU.

$ qemu-system-arm -M \?

2.3 Download U-Boot

Fetch the current Das U-Boot source code through git.

$ git clone git://git.denx.de/u-boot.git

Or download a released version from ftp site.

2.4 Compile U-Boot.

$ cd u-boot/
$ make versatileqemu_config
$ make

Or if a cross toolchain with different prefix is used, ELDK's "arm-linux-gnueabi-" for example,

$ make CROSS_COMPILE=arm-linux-gnueabi-

3 Testing and Debugging

3.1 Normal Mode

We can execute QEMU to run U-Boot for ARM, then develop and debug as we wish.

$ qemu-system-arm -nographic -M versatilepb -m 128 -kernel u-boot
U-Boot 2011.12-00046-gbfcc40b-dirty (Dec 26 2011 - 18:11:37)

DRAM:  128 MiB
WARNING: Caches not enabled
Using default environment

In:    serial
Out:   serial
Err:   serial
Net:   SMC91111-0
Warning: failed to set MAC address

VersatilePB #

A list of u-boot commands and environment variables,

VersatilePB # help
?       - alias for 'help'
base    - print or set address offset
bdinfo  - print Board Info structure
bootm   - boot application image from memory
bootp   - boot image via network using BOOTP/TFTP protocol
cmp     - memory compare
cp      - memory copy
crc32   - checksum calculation
dhcp    - boot image via network using DHCP/TFTP protocol
env     - environment handling commands
erase   - erase FLASH memory
flinfo  - print FLASH memory information
go      - start application at address 'addr'
help    - print command description/usage
iminfo  - print header information for application image
loop    - infinite loop on address range
md      - memory display
mm      - memory modify (auto-incrementing address)
mtest   - simple RAM read/write test
mw      - memory write (fill)
nm      - memory modify (constant address)
ping    - send ICMP ECHO_REQUEST to network host
printenv- print environment variables
protect - enable or disable FLASH write protection
reset   - Perform RESET of the CPU
setenv  - set environment variables
tftpboot- boot image via network using TFTP protocol
version - print monitor, compiler and linker version
VersatilePB #
VersatilePB # printenv
baudrate=38400
bootargs=root=/dev/nfs mem=128M ip=dhcp netdev=25,0,0xf1010000,0xf1010010,eth0
bootdelay=2
bootfile=/tftpboot/uImage
ethact=SMC91111-0
stderr=serial
stdin=serial
stdout=serial
verify=n

Environment size: 219/8188 bytes
VersatilePB #

3.2 Debugging with GDB

While QEMU's built-in monitor provides limited debugging support, it can act as a remote debugging target for GDB.

$ qemu-system-arm -nographic -M versatilepb -m 128 -kernel u-boot -S -s

'-S' tells QEMU not to start CPU at startup. We must type 'c' in monitor to let it continue. '-s' is shorthand for '-gdb tcp::1234', telling QEMU to wait for gdb connection on TCP port 1234.

On the other hand, we start the ARM GDB and set target to TCP port 1234 as well.

$ arm-linux-gnueabi-gdb u-boot
GNU gdb (GDB) 7.3
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "--host=i686-eldk-linux --target=arm-linux-gnueabi".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /home/winfred/u-boot/u-boot...done.
(gdb) target remote :1234
Remote debugging using :1234
_start () at start.S:64
64              b       reset
(gdb)

We are able to use GDB for source level debugging now.

(gdb) n
188             mrs     r0,cpsr
(gdb) n
189             bic     r0,r0,#0x1f
(gdb) b board_init_f
Breakpoint 1 at 0x10b70: file board.c, line 270.
(gdb) c
Continuing.

Breakpoint 1, board_init_f (bootflag=0) at board.c:270
270             gd = (gd_t *) ((CONFIG_SYS_INIT_SP_ADDR) & ~0x07);
(gdb)

3.3 Debugging After Relocation

In order to debug U-Boot after relocation, we need to know the address which U-Boot relocates itself to. There are couple of ways to get it,

Get it from U-Boot 'bdinfo' command.

VersatilePB # bdinfo
arch_number = 0x00000183
boot_params = 0x00000100
DRAM bank   = 0x00000000
-> start    = 0x00000000
-> size     = 0x08000000
ethaddr     = (not set)
ip_addr     = 0.0.0.0
baudrate    = 38400 bps
TLB addr    = 0x07FF0000
relocaddr   = 0x07FC5000
reloc off   = 0x07FB5000
irq_sp      = 0x07FA2F70
sp start    = 0x07FA2F60
FB base     = 0x00000000
VersatilePB #

Print the value of gd->relocaddr. Since gd is avalable after board_init_f(), and it is storeed in register $r1,

(gdb) b relocate_code
Breakpoint 1 at 0x1007c: file start.S, line 226.
(gdb) c
Continuing.

Breakpoint 1, relocate_code () at start.S:226
226             mov     r5, r1  /* save addr of gd */
(gdb) p/x ((gd_t *)$r1)->relocaddr
$1 = 0x7fc5000
(gdb)

Having the relocated address, we need to reload the symbol file accordingly.

(gdb) d
Delete all breakpoints? (y or n) y
(gdb) symbol-file
Discard symbol table from `/home/winfred/u-boot/u-boot'? (y or n) y
No symbol file now.
(gdb) add-symbol-file u-boot 0x7fc5000
add symbol table from file "u-boot" at
.text_addr = 0x7fc5000
(y or n) y
Reading symbols from /home/winfred/u-boot/u-boot...done.
(gdb)

Then, debug U-Boot after relocation. Let's try to break at the first function after the relocation,

(gdb) b board_init_r
Breakpoint 2 at 0x7fc5d64: file board.c, line 455.
(gdb) c
Continuing.

Breakpoint 2, board_init_r (id=0x7fa2f70, dest_addr=133976064) at board.c:455
455             gd = id;
(gdb)

Happy debugging.

4 Reference

Date: 2011-12-26 Thu

HTML generated by org-mode 6.33x in emacs 23

MIPS Exceptions Initialization and Handling on Linux

2011-04-06T04:32:00.000-07:00

Exceptions and interrupts are un-expected events that disrupt the normal flow of execution. Interrupts are unexpected events from outside the CPU core. Exceptions are from within the CPU, including memory translation exceptions, cache misses, unusual program conditions such as unaligned loads, system calls and traps, and so on.

MIPS architecture adopts precise exception mechanisms, and make it easy for the software to deal with the exceptions. After any exception, CP0 EPC register points to the correct place to restart execution after the exception is dealt with.

The followed is a brief list of MIPS exceptions in order of their relative priority.

Reset
NMI -- non-maskable interrupts
Interrupt -- hardware and software interrupts
AdEL -- fetch address alignment error; fetch reference to protected address
TLBL -- fetch TLB miss
ICache Error
Sys -- execution of SYSCALL instruction
Ov -- execution of an arithmetic instruction that overflowed
Tr -- execution of a trap
AdEL -- load address alignement error; load reference to protected address
AdES -- store address alignement error; store reference to protected address
TLBL -- load TLB miss
TLBS -- store TLB miss
DCache Error

MIPSr2 architecture supports 3 interrupt modes. It provides interrupt compatibility mode, which behaves identical to MIPSr1. It also supports vectored interrupt (VI) mode, and permits the use of an external interrupt controller (EIC).

VI mode adds the ability to prioritize and vector interrupts to a handler dedicated to that interrupt. Each interrupt starts at the address according to the interrupt signal. It also allows us to configure the space between different entry points (VS of IntCtl register). In EIC mode, the six independent signals become a 6-bit binary number: Zero means no interrupt, others are 63 distinct interrupt codes.

An exception vector is where the exception handling starts. MIPS exceptions are handled either through the general exception vector (offset 0x180), or the special interrupt vector (0x200), based on the value of IV of Cause register. Software was required to prioritize exceptions in the handler prologue.

The followed is the table of exception entry points (vector addresses).

Where Base is 0x80000000 by default, and can be configured by VA of EBase register. And RBase is 0xBFC00000 by default, and can be changed by configuring SI_UseExceptionBase to 1. Cache error handling instructions have to be fetched through uncached KSeg1 window.

Here is what a MIPS CPU does when it decides to take an exception:

It sets up the EPC to point to the restart location.
It force CPU into kernel mode and disables interrupts. (by setting EXL of SR register)
Cause register is set up so that software can see the reason for the exception. On address exceptions, BadVAddr is also set. Memory management system exceptions set up some of the MMU registers too.
CPU then starts fetching instructions from the exception entry point, and everything else is up to software.

So, let's see how MIPS Linux initializes handling routines and handles exceptions.

  1 void __init trap_init(void)
  2 {
  3         extern char except_vec3_generic, except_vec3_r4000;
  4         extern char except_vec4;
  5         unsigned long i;
  6         int rollback;
  7 
  8         check_wait();
  9         rollback = (cpu_wait == r4k_wait);
 10 
 11 #if defined(CONFIG_KGDB)
 12         if (kgdb_early_setup)
 13                 return;       /* Already done */
 14 #endif
 15 
 16         if (cpu_has_veic || cpu_has_vint) {
 17                 unsigned long size = 0x200 + VECTORSPACING*64;
 18                 ebase = (unsigned long)
 19                         __alloc_bootmem(size, 1 << fls(size), 0);
 20         } else {
 21                 ebase = CKSEG0;
 22                 if (cpu_has_mips_r2)
 23                         ebase += (read_c0_ebase() & 0x3ffff000);
 24         }
 25 
 26         per_cpu_trap_init();
 27 
 28         /*
 29          * Copy the generic exception handlers to their final destination.
 30          * This will be overriden later as suitable for a particular
 31          * configuration.
 32          */
 33         set_handler(0x180, &except_vec3_generic, 0x80);
 34

line 20~24: It initializes base address of exception vectors (ebase) to 0x80000000 (CKSEG0), the power-on default value. MIPSr2 provides an option to adjust ebase according to VA (bit 29:12) of EBase register.

line 16~19: If VI mode or EIC mode is used, bootmem from offset 0 (goal 0) will be allocated for the handler routeines. __alloc_bootmem returns virtual address which will also be 0x80000000 (an article FYR). There are 64 handlers in total, and each is of size 0x100 by default. The first 0x200 is also reserved for TLB Refill and Cache Error handlers.

line 26: Cache Error handler and TLB Refill handler are configured in per_cpu_trap_init().

line 33: Default exception handler would be located 0x80000180 and of size 128 bytes. set_handler() copies 128 bytes from the location of except_vec3_generic to ebase + 0x180.

Take a look at per_cpu_trap_init() and except_vec3_generic.

  1 void __cpuinit per_cpu_trap_init(void)
  2 {
  3         unsigned int cpu = smp_processor_id();
  x         ......
 50         if (cpu_has_veic || cpu_has_vint) {
 51                 unsigned long sr = set_c0_status(ST0_BEV);
 52                 write_c0_ebase(ebase);
 53                 write_c0_status(sr);
 54                 /* Setting vector spacing enables EI/VI mode  */
 55                 change_c0_intctl(0x3e0, VECTORSPACING);
 56         }
 57         if (cpu_has_divec) {
 58                 if (cpu_has_mipsmt) {
 59                         unsigned int vpflags = dvpe();
 60                         set_c0_cause(CAUSEF_IV);
 61                         evpe(vpflags);
 62                 } else
 63                         set_c0_cause(CAUSEF_IV);
 64         }
  x         ......
 72         if (cpu_has_mips_r2) {
 73                 cp0_compare_irq_shift = CAUSEB_TI - CAUSEB_IP;
 74                 cp0_compare_irq = (read_c0_intctl() >> INTCTLB_IPTI) & 7;
 75                 cp0_perfcount_irq = (read_c0_intctl() >> INTCTLB_IPPCI) & 7;
 76                 if (cp0_perfcount_irq == cp0_compare_irq)
 77                         cp0_perfcount_irq = -1;
 78         } else {
 79                 cp0_compare_irq = CP0_LEGACY_COMPARE_IRQ;
 80                 cp0_compare_irq_shift = cp0_compare_irq;
 81                 cp0_perfcount_irq = -1;
 82         }
  x         ......
 99                 cpu_cache_init();
100                 tlb_init();
  x         ......
111 }

line 50~56: It sets BEV for Status register, configure EBase register with ebase, and configure VS in IntCtrl register. VS will be VECTORSPACING(0x100) << 4 = 8 to get a 256 bytes spacing.

line 57~64: It sets IV in Cause register and special interrupt vector will be used instead.

line 72~82: MIPSr2 provides the flexibility to change the default interrupt numbers for timer interrupt and performance counter interrupt.

line 99: It calls xxx_cache_init() accordingly, r4k_cache_init() for example, and calls set_uncached_handler() to build the Cache Error exception handler. It copies the handler except_vec2_generic to ebase + 0x100. set_uncached_handler() is similar to set_handler(), the only difference is that it uses KSeg1 address of ebase. except_vec2_generic turns off KSeg0 caching (sets K0 to 2 in Config register), and jump to cache_parity_error to handle the exception.

line 100: It calls tlb_init(), then build_tlb_refill_handler(), then build handlers handling exceptions including TLB refill exception, TLB load exception (handle_tlbl()), TLB store exception (handle_tlbs()), and TLB modify read-only area exception (handle_tlbm()). The build_r4000_tlb_refill_handler() builds tlb_handler with correct instructions, copy to final_handler, fixes relocations, and copy final_handler to ebase + 0.

And except_vec3_generic,

 1 NESTED(except_vec3_generic, 0, sp)
 2         .set    push
 3         .set    noat
 4 #if R5432_CP0_INTERRUPT_WAR
 5         mfc0    k0, CP0_INDEX
 6 #endif
 7         mfc0    k1, CP0_CAUSE
 8         andi    k1, k1, 0x7c
 9 #ifdef CONFIG_64BIT
10         dsll    k1, k1, 1
11 #endif
12 PTR_L   k0, exception_handlers(k1)
13         jr      k0
14         .set    pop
15         END(except_vec3_generic)

line 2: save the current status of flags
line 3: prevent the assembler from using AT register
line 7~8: read ExeCode (bit 6:2) from Cause register

line 10: In 64-bit environment, pointers are 8-byte aligned, which is double of size compared to a 4-byte pointer.

line 12: load the handler address from exception_handlers + k1. exception_handlers is an array of pointers storing address of exception handlers.

line 13: jump to the correct exception handler (according to the ExeCode)
line 14: restore the previous status of flags

Let us leave per_cpu_trap_init() and except_vec3_generic() there and go back to trap_init().

  1 void __init trap_init(void)
  2 {
  x         ......
 35         /*
 36          * Setup default vectors
 37          */
 38         for (i = 0; i <= 31; i++)
 39                 set_except_vector(i, handle_reserved);
 40 
 41         /*
 42          * Copy the EJTAG debug exception vector handler code to it's final
 43          * destination.
 44          */
 45         if (cpu_has_ejtag && board_ejtag_handler_setup)
 46                 board_ejtag_handler_setup();
 47 
 48         /*
 49          * Only some CPUs have the watch exceptions.
 50          */
 51         if (cpu_has_watch)
 52                 set_except_vector(23, handle_watch);
 53

line 38~39: set the 32 exception handlers to handle_reserved() by default. set_except_vector() will substitute the i-th entry of exception_handlers[] with the given address.

line 44~45: configure the EJTAG debug exception handler if needed
line 51~52: configure watch exception handler to handle_watch()

handle_reserved() and handle_watch() are both built in arch/mips/kernel/genex.S as the followed:

 1         BUILD_HANDLER adel ade ade silent               /* #4  */
 2         BUILD_HANDLER ades ade ade silent               /* #5  */
 3         BUILD_HANDLER ibe be cli silent                 /* #6  */
 4         BUILD_HANDLER dbe be cli silent                 /* #7  */
 5         BUILD_HANDLER bp bp sti silent                  /* #9  */
 6         BUILD_HANDLER ri ri sti silent                  /* #10 */
 7         BUILD_HANDLER cpu cpu sti silent                /* #11 */
 8         BUILD_HANDLER ov ov sti silent                  /* #12 */
 9         BUILD_HANDLER tr tr sti silent                  /* #13 */
10         BUILD_HANDLER fpe fpe fpe silent                /* #15 */
11         BUILD_HANDLER mdmx mdmx sti silent              /* #22 */
12 #ifdef  CONFIG_HARDWARE_WATCHPOINTS
13         /*
14          * For watch, interrupts will be enabled after the watch
15          * registers are read.
16          */
17         BUILD_HANDLER watch watch cli silent            /* #23 */
18 #else
19         BUILD_HANDLER watch watch sti verbose           /* #23 */
20 #endif
21         BUILD_HANDLER mcheck mcheck cli verbose         /* #24 */
22         BUILD_HANDLER mt mt sti silent                  /* #25 */
23         BUILD_HANDLER dsp dsp sti silent                /* #26 */
24         BUILD_HANDLER reserved reserved sti verbose     /* others */

Each call to BUILD_HANDLER will build two functions, handle_\exception and handle_\exception_int. Thus, you may look the symbol table for handle_adel(), handle_adel_int(), handle_ades(), handle_ades_int(), and so on.

How do these handlers get prepared and handle exceptions?

 1         .macro  __BUILD_HANDLER exception handler clear verbose ext
 2         .align  5
 3         NESTED(handle_\exception, PT_SIZE, sp)
 4         .set    noat
 5         SAVE_ALL
 6         FEXPORT(handle_\exception\ext)
 7         __BUILD_clear_\clear
 8         .set    at
 9         __BUILD_\verbose \exception
10         move    a0, sp
11         PTR_LA  ra, ret_from_exception
12         j       do_\handler
13         END(handle_\exception)
14         .endm
15 
16         .macro  BUILD_HANDLER exception handler clear verbose
17         __BUILD_HANDLER \exception \handler \clear \verbose _int
18         .endm

line 16~18: BUILD_HANDLER invokes __BUILD_HANDLER to build handlers

line 3: Frame size of handle_\exception() is PT_SIZE, which is automatically generated by Kbuild according to sizeof(struct pt_regs). pt_regs defines how registers are stored in stack during the exception.

line 4: tell the assembler no to use AT register because we will

line 5: save some CP0 registers and all the general purpose registers by calling SAVE_SOME, SAVE_AT, SAVE_TEMP, and SAVE_STATIC. The registers will be stored onto stack in order as defined in pt_regs.

line 6: export the handle_\exception_int() symbol. handle_\exception() and handle_\exception_int() share contents below this line.

line 7: could be CLI, STI, fpe, or ade
line 8: The assembler is free to use AT because we've done our saving job.
line 9: print the information of EPC if verbose is set

line 10~11: prepare to jump do_\handler(). The first argument is sp, the pointer to pt_regs. The return address is the address of ret_from_exception(), the function called when exception handling is done. ret_from_exception() will restore registers back, and return to the original address (discussed later).

line 12: call the exception handling function do_\handler(), such as do_ade(), do_be(), do_ri, and so on.

Okey, let's go back to trap_init() again.

  1 void __init trap_init(void)
  2 {
  x         ......
 54         /*
 55          * Initialise interrupt handlers
 56          */
 57         if (cpu_has_veic || cpu_has_vint) {
 58                 int nvec = cpu_has_veic ? 64 : 8;
 59                 for (i = 0; i < nvec; i++)
 60                         set_vi_handler(i, NULL);
 61         }
 62         else if (cpu_has_divec)
 63                 set_handler(0x200, &except_vec4, 0x8);
 64 
 65         /*
 66          * Some CPUs can enable/disable for cache parity detection, but does
 67          * it different ways.
 68          */
 69         parity_protection_init();
 70 
 71         /*
 72          * The Data Bus Errors / Instruction Bus Errors are signaled
 73          * by external hardware.  Therefore these two exceptions
 74          * may have board specific handlers.
 75          */
 76         if (board_be_init)
 77                 board_be_init();
 78

line 57~61: in VI or EIC mode, set interrupt handler to NULL by default. BSP providers need to configure every interrupt handler accordingly. It calls set_vi_srs_handler() and copies the given handler to ebase + 0x200 + n*VECTORSPACING.

line 62~63: configure except_vec4 as the default special interrupt handler
line 69: enable cache parity dectection if it is implemented. (ErrCtrl register)
line 76~77: configure the Data/Instruction Bus Error exception handler if supported

  1 void __init trap_init(void)
  2 {
  x         ......
 79         set_except_vector(0, rollback ? rollback_handle_int : handle_int);
 80         set_except_vector(1, handle_tlbm);
 81         set_except_vector(2, handle_tlbl);
 82         set_except_vector(3, handle_tlbs);
 83 
 84         set_except_vector(4, handle_adel);
 85         set_except_vector(5, handle_ades);
 86 
 87         set_except_vector(6, handle_ibe);
 88         set_except_vector(7, handle_dbe);
 89 
 90         set_except_vector(8, handle_sys);
 91         set_except_vector(9, handle_bp);
 92         set_except_vector(10, rdhwr_noopt ? handle_ri :
 93                           (cpu_has_vtag_icache ?
 94                            handle_ri_rdhwr_vivt : handle_ri_rdhwr));
 95         set_except_vector(11, handle_cpu);
 96         set_except_vector(12, handle_ov);
 97         set_except_vector(13, handle_tr);
  x         ......
114         if (board_nmi_handler_setup)
115                 board_nmi_handler_setup();
116 
117         if (cpu_has_fpu && !cpu_has_nofpuex)
118                 set_except_vector(15, handle_fpe);
119 
120         set_except_vector(22, handle_mdmx);
121 
122         if (cpu_has_mcheck)
123                 set_except_vector(24, handle_mcheck);
124 
125         if (cpu_has_mipsmt)
126                 set_except_vector(25, handle_mt);
127 
128         set_except_vector(26, handle_dsp);
129

line 79~128: configure exception handler seperately. 0~26 are exception codes (ExeCode of Cause register), the meaning of their value could be referenced to MIPS user manual. For example, 0 means that it is an interrupt, 1 means that TLB modify to read-only page exception.

  1 void __init trap_init(void)
  2 {
  x         ......
130         if (cpu_has_vce)
131                 /* Special exception: R4[04]00 uses also the divec space. */
132                 memcpy((void *)(ebase + 0x180), &except_vec3_r4000, 0x100);
133         else if (cpu_has_4kex)
134                 memcpy((void *)(ebase + 0x180), &except_vec3_generic, 0x80);
135         else
136                 memcpy((void *)(ebase + 0x080), &except_vec3_generic, 0x80);
137 
138         local_flush_icache_range(ebase, ebase + 0x400);
139         flush_tlb_handlers();
140 
141         sort_extable(__start___dbe_table, __stop___dbe_table);
142 
143         cu2_notifier(default_cu2_call, 0x80000000);    /* Run last  */
144 }

line 130~136: Here we don't support virtual coherence conflict exception, and copy except_vec3_generic to the address of default exception vector again.

line 138~139: flush ICache for address in the range of exception handlers
line 141: sort the exception table for data bus error

Interrupt is a special type of the exceptions; it comes from outside the CPU core. When a exception with ExeCode equal to 0, it is an interrupt. Let's see how interrupts are dealt with:

 1 NESTED(handle_int, PT_SIZE, sp)
 2 +-- 31 lines: #ifdef CONFIG_TRACE_IRQFLAGS------------------
 3         SAVE_ALL
 4         CLI
 5         TRACE_IRQS_OFF
 6 
 7         LONG_L  s0, TI_REGS($28)
 8         LONG_S  sp, TI_REGS($28)
 9         PTR_LA  ra, ret_from_irq
10         j       plat_irq_dispatch
11         END(handle_int)

line 2, 5: trace IRQ stuff, we are not going to discuss it here.
line 3: save some CP0 registers and all the general purpose registers to stack
line 4: CLI to clear the interrupts

line 7: save the information of pt_regs pointer to current thread_info to s0
line 8: load the pt_regs pointer with the stack pointer (where we saved all registers)

line 9: set return address to ret_from_irq instead, so that handler will return to ret_from_irq after the interrupt

line 10: jump to the platform dependent interrupt handling routine

As you may see, handle_int is similar to handle_\exception with the differences:

handle_int needs to CLI, while handle_\exception may use CLI, STI, fpe, or ade.
the return address of handle_int is ret_from_irq; the return address of handle_\exception is ret_from_exception
handle_int will jump to plat_irq_dispatch, handle_\exception will jump to do_\handler
do_\handler has an argument, a pointer to pt_regs. handle_\exception copies stack pointer to a0 for the usage.
plat_irq_dispatch has no argument, and the pointer to pt_regs will be in current thread_info. So handle_int has to save the current pt_regs to s0 before updating it with stack pointer

Interrupts and handling are often different between platforms. Developers have to implement plat_irq_dispatch() accordingly. A common handler is provded in arch/mips/mipssim/sim_int.c. It finds the first pending interrupt number and calls do_IRQ(no.) to handle it.

 1 asmlinkage void plat_irq_dispatch(void)
 2 {
 3         unsigned int pending = read_c0_cause() & read_c0_status() & ST0_IM;
 4         int irq;
 5 
 6         irq = irq_ffs(pending);
 7 
 8         if (irq > 0)
 9                 do_IRQ(MIPS_CPU_IRQ_BASE + irq);
10         else
11                 spurious_interrupt();
12 }

line 3: read pending interrupts from Cause register and masked by IM7-0 of Status register

line 6: find the first bit set in pendings. This is an example of handling compatible interrupts, first bit is the highest priority interrupt.

line 8~9: do_IRQ() to handle the interrupt
line 10~11: It is a spurious interrupt (should not happen).

And when exception or interrupt has been handle, it invokes ret_from_exception or ret_from_irq to return to the original executions.

 1 #ifndef CONFIG_PREEMPT
 2 #define resume_kernel   restore_all
 3 #else
 4 #define __ret_from_irq  ret_from_exception
 5 #endif
 6 
 7         .text
 8         .align  5
 9 #ifndef CONFIG_PREEMPT
10 FEXPORT(ret_from_exception)
11         local_irq_disable                       # preempt stop
12         b       __ret_from_irq
13 #endif
14 FEXPORT(ret_from_irq)
15         LONG_S  s0, TI_REGS($28)
16 FEXPORT(__ret_from_irq)
17         LONG_L  t0, PT_STATUS(sp)               # returning to kernel mode?
18         andi    t0, t0, KU_USER
19         beqz    t0, resume_kernel
20 
21 resume_userspace:
22         local_irq_disable               # make sure we dont miss an
23                                         # interrupt setting need_resched
24                                         # between sampling and return
25         LONG_L  a2, TI_FLAGS($28)       # current->work
26         andi    t0, a2, _TIF_WORK_MASK  # (ignoring syscall_trace)
27         bnez    t0, work_pending
28         j       restore_all

line 10,14: ret_from_exception and ret_from_irq share the context because they are similar.
line 15: ret_from_irq has to restore back the pointer of pt_regs of thread_info stored in s0 previously.
line 17~18: read UM of Status register and see if it is in kernel mode
line 19, 1~3: if in kernel mode, jump to resume_kernel (it actually calls restore_all)
line 21~28: if in user mode, also call restore_all after finishing pending workings

restore_all will call RESTORE_TEMP, RESTORE_AT, RESTORE_STATIC, RESTORE_SOME, and RESTORE_SP_AND_RET. RESTORE_SP_AND_RET will execute eret instruction finally, it clears EXL of Status register and returns to the address stored in EPC, then original execution continues.

References:

Dominic Sweetman, See MIPS Run
MIPS Software User Manual
Exception and Interrupt handling in the MIPS architecture
TLDP, David A Rusling, Interrupts and Interrupt Handling
the Linux Cross Reference
Linux MIPS

Learning the Linux Boot Memory Allocator (MIPS)

2011-03-04T00:10:00.000-08:00

Do you ever look at the report of 'cat /proc/meminfo' and wonder why the reported MemTotal is not equal to the amount of memory you have for the system? The followed is an example report from one of my systems. It shows the total memory is 60860kB (59.4MB) from line 2:

1 # cat /proc/meminfo
2 MemTotal:          60860 kB
3 MemFree:           47448 kB
4 Buffers:               0 kB
5 Cached:             7180 kB
6 SwapCached:            0 kB
7 ...

which is different from the amount of memory that bootloader gave to kernel, 64MB.

1 ...
2 ## Transferring control to Linux (at address 800042f0) ...
3 ## Giving linux memsize in MB, 64

The total physical memory is 64MB (0x4000000) according to the kernel log line 4. And the memory available (line 20) is 59176k because 6360k bytes is reserved. As we know that available = total - reserved. (59176k = 65536k - 6360k).

 1 # dmesg
 2 ...
 3 Determined physical RAM map:
 4  memory: 04000000 @ 00000000 (usable)
 5 Initrd not found or empty - disabling initrd
 6 Zone PFN ranges:
 7   Normal   0x00000000 -> 0x00004000
 8 Movable zone start PFN for each node
 9 early_node_map[1] active PFN ranges
10     0: 0x00000000 -> 0x00004000
11 Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 16256
12 Kernel command line: console=ttyS1,57600n8 root=/dev/ram0 rootfstype=squashfs
13 PID hash table entries: 256 (order: -2, 1024 bytes)
14 Dentry cache hash table entries: 8192 (order: 3, 32768 bytes)
15 Inode-cache hash table entries: 4096 (order: 2, 16384 bytes)
16 Primary instruction cache 64kB, VIPT, 4-way, linesize 32 bytes.
17 Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
18 Writing ErrCtl register=00000000
19 Readback ErrCtl register=00000000
20 Memory: 59176k/65536k available (2905k kernel code, 6360k reserved, 980k data, 1684k init, 0k highmem)
21 ...
22 Freeing unused kernel memory: 1684k freed
23 ...

However, you may notice that MemTotal and the available memory are not equal. It is because kernel can free memory reserved for initialization during booting, if the memory is not going to used again. Developers mark those codes with a specific attribute, and link them into the .init section. Kernel will release the memory by invoking free_initmem() when initial bootup is done.

Thus, MemTotal = total - reserved + .init
In the example, 60860k = 65536k - 6360k + 1684k

The question is what does kernel reserved those memory for? Most of reserved memory is used to store kernel image. Take a look to the information read from vmlinux as the followed, sections with number within 0 and 13 are required, including the well known text, data, and bss sections. We can conclude that more than 0x5a7b90 bytes of memory could be used. The minimum unit of reserved memory is 1 page (4K bytes by default). Hense 0x5a7b90 will be 4K aligned instead, which is 0x5a8000.

 1 $ readelf -S vmlinux
 2 There are 19 section headers, starting at offset 0x576694:
 3 
 4 Section Headers:
 5   [Nr] Name              Type            Addr     Off    Size   ES Flg Lk Inf Al
 6   [ 0]                   NULL            00000000 000000 000000 00      0   0  0
 7   [ 1] .text             PROGBITS        80000000 002000 2d66f8 00  AX  0   0 2048
 8   [ 2] __ex_table        PROGBITS        802d6700 2d8700 001b20 00   A  0   0  4
 9   [ 3] .rodata           PROGBITS        802d9000 2db000 0b09fc 00   A  0   0 32
10   [ 4] __ksymtab         PROGBITS        803899fc 38b9fc 004d20 00   A  0   0  4
11   [ 5] __ksymtab_gpl     PROGBITS        8038e71c 39071c 0022c8 00   A  0   0  4
12   [ 6] __ksymtab_strings PROGBITS        803909e4 3929e4 00f55e 00   A  0   0  1
13   [ 7] __param           PROGBITS        8039ff44 3a1f44 0010bc 00   A  0   0  4
14   [ 8] .data             PROGBITS        803a2000 3a4000 0299c0 00  WA  0   0 8192
15   [ 9] .data..shared_ali PROGBITS        803cb9c0 3cd9c0 000080 00  WA  0   0 32
16   [10] .init.text        PROGBITS        803cc000 3ce000 024270 00  AX  0   0  4
17   [11] .init.data        PROGBITS        803f0270 3f2270 17f6f2 00  WA  0   0  8
18   [12] .exit.text        PROGBITS        8056f964 571964 0015d8 00  AX  0   0  4
19   [13] .bss              NOBITS          80571000 572f3c 036b90 00  WA  0   0 4096
20   [14] .mdebug.abi32     PROGBITS        805a7b90 572f3c 000000 00      0   0  1
21   [15] .comment          PROGBITS        00000000 572f3c 0036a2 00      0   0  1
22   [16] .shstrtab         STRTAB          00000000 5765de 0000b3 00      0   0  1
23   [17] .symtab           SYMTAB          00000000 57698c 063290 10     18 18308  4
24   [18] .strtab           STRTAB          00000000 5d9c1c 077ce3 00      0   0  1
25 Key to Flags:
26   W (write), A (alloc), X (execute), M (merge), S (strings)
27   I (info), L (link order), G (group), x (unknown)
28   O (extra OS processing required) o (OS specific), p (processor specific)

To dig deeper, let's turn the bootmem debugging on by adding "bootmem_debug=1" to the kernel command line. We focus on MIPS in this example, and implementation of bootmem allocator for other architectures could be slightly different. But the basic idea is the same. Bootmem allocator uses a bitmap structure representing the use of memory. It allows us to allocate memory by simply setting or clearing the corresponding bit, before getting the zone allocator ready.

 1 # dmesg
 2 ...
 3 Determined physical RAM map:
 4  memory: 04000000 @ 00000000 (usable)
 5 bootmem::init_bootmem_core nid=0 start=0 map=5a8 end=4000 mapsize=800
 6 bootmem::mark_bootmem_node nid=0 start=5a8 end=4000 reserve=0 flags=0
 7 bootmem::__free nid=0 start=5a8 end=4000
 8 bootmem::mark_bootmem_node nid=0 start=5a8 end=5a9 reserve=1 flags=0
 9 bootmem::__reserve nid=0 start=5a8 end=5a9 flags=0
10 Initrd not found or empty - disabling initrd
11 Zone PFN ranges:
12   Normal   0x00000000 -> 0x00004000
13 Movable zone start PFN for each node
14 early_node_map[1] active PFN ranges
15     0: 0x00000000 -> 0x00004000
16 bootmem::alloc_bootmem_core nid=0 size=80000 [128 pages] align=20 goal=1000000 limit=0
17 bootmem::__reserve nid=0 start=1000 end=1080 flags=1
18 bootmem::alloc_bootmem_core nid=0 size=8 [1 pages] align=20 goal=1000000 limit=0
19 bootmem::__reserve nid=0 start=1080 end=1081 flags=1
20 bootmem::alloc_bootmem_core nid=0 size=200 [1 pages] align=20 goal=1000000 limit=0
21 bootmem::__reserve nid=0 start=1081 end=1081 flags=1
22 bootmem::alloc_bootmem_core nid=0 size=1c [1 pages] align=20 goal=1000000 limit=0
23 bootmem::__reserve nid=0 start=1081 end=1081 flags=1
24 bootmem::alloc_bootmem_core nid=0 size=49 [1 pages] align=20 goal=1000000 limit=0
25 bootmem::__reserve nid=0 start=1081 end=1081 flags=1
26 bootmem::alloc_bootmem_core nid=0 size=49 [1 pages] align=20 goal=1000000 limit=0
27 bootmem::__reserve nid=0 start=1081 end=1081 flags=1
28 Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 16256
29 Kernel command line: console=ttyS1,57600n8 root=/dev/ram0 rootfstype=squashfs bootmem_debug=1
30 bootmem::alloc_bootmem_core nid=0 size=400 [1 pages] align=20 goal=1000000 limit=0
31 bootmem::__reserve nid=0 start=1081 end=1081 flags=1
32 PID hash table entries: 256 (order: -2, 1024 bytes)
33 bootmem::alloc_bootmem_core nid=0 size=8000 [8 pages] align=20 goal=1000000 limit=0
34 bootmem::__reserve nid=0 start=1081 end=1089 flags=1
35 Dentry cache hash table entries: 8192 (order: 3, 32768 bytes)
36 bootmem::alloc_bootmem_core nid=0 size=4000 [4 pages] align=20 goal=1000000 limit=0
37 bootmem::__reserve nid=0 start=1089 end=108d flags=1
38 Inode-cache hash table entries: 4096 (order: 2, 16384 bytes)
39 Primary instruction cache 64kB, VIPT, 4-way, linesize 32 bytes.
40 Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
41 Writing ErrCtl register=00000000
42 Readback ErrCtl register=00000000
43 bootmem::free_all_bootmem_core nid=0 start=0 end=4000 aligned=1
44 bootmem::free_all_bootmem_core nid=0 released=39cb
45 Memory: 59176k/65536k available (2905k kernel code, 6360k reserved, 980k data, 1684k init, 0k highmem)
46 ...
47 Freeing unused kernel memory: 1684k freed
48 ...

bootmem_init() initialises the bootmem allocator and setup initrd related data if needed. It calls init_bootmem_node() and calls init_bootmem_core(), and reserves all the pages intaially. In function bootmem_init(), the variable mapstart comes from the symbol _end. It is the first free PFN (page frame number) and states the location of bootmem bitmap.

the _end from symbol table:

 1 $ tail System.map
 2 805a7ab0 b sit_net_id
 3 805a7ac0 b ip6_tnl_net_id
 4 805a7ad0 B br_should_route_hook
 5 805a7ae0 b br_fdb_cache
 6 805a7ae4 b fdb_salt
 7 805a7af0 b br_mac_zero_aligned
 8 805a7b00 B vlan_net_id
 9 805a7b04 b vlan_group_hash
10 805a7b90 A __bss_stop
11 805a7b90 A _end

Line 5 of kernel log tells us that: it iniailises node ID 0 (This is an UMA system and only 1 node exists.), PFN range is from 0 to 0x3999, and the bootmem bitmap is located at PFN 0x5a8 and of size 0x800 bytes. 64MB memory needs pages of number 64M/4K = 16K = 0x4000, and 16KB pages needs bootmem bitmap of size 16K/8 = 2K = 0x800 bytes.

Line 6~7: bootmem_init() then calls free_bootmem() and calls mark_bootmem(). It marks pages after the location of kernel image usable, and the ranage is within PFN 0x5a8 and 0x3999.

Line 8~9: bootmem_init() then calls reserve_bootmem() and calls mark_bootmem(). It reserve memory for bootmem bitmap. The bootmap size is 0x800 bytes, less than 1 page, so PFN 0x5a8 is enough.

Line 11~15: kernel initialises paging system, calls free_area_init_nodes() to initialise all pg_data_t and zone data, and prints information of all zones, movable zones, and early node map.

Line 16~17: free_area_init_nodes() calls free_area_init_node() for each node. It calculate total pages of this node, and calls alloc_node_mem_map() to allocate memory for memory map. The size of one page structure is 32 bytes, and we need 0x4000 pages totally. So the allocation size would be 0x32 * 0x4000 = 0x80000 bytes, which is 128 pages (0x80000 / 0x1000). The goal 0x1000000 means that it allocates memory in normal zone. It skips the first 16MB of DMA addresses. Because the address 0x1000000 is PFN 0x1000, alloc_bootmem_core() will allocate pages starting from PFN 0x1000.

Line 18~21: free_area_init_node() calls free_area_init_core() to built the memory map, and initialise freelists and buddy bitmaps of every zone in the node. It calls setup_usemap() to allocate bootmem for pageblock_flags. The usemap size is 0x4000 / 0x400 * 3, rounded up 32, then / 8 = 8 bytes (line 18). During zone data initialization, free_area_init_core() also calls zone_wait_table_init() to initialise wait queue hash table. Size of wait_queue_head_t is 8 bytes and number of wait queue is 64, so the allocation size would be 8 * 0x40 = 0x200 bytes (line 20).

Line 21: Why would the start and end PFN be the same? Line 19 reserves less than 1 page, and alloc_bootmem_core() will attemp to use a free fragment of the last allocated page. It calculates allocation size each time called, and sums up the information in bdata->last_end_off. When last_end_off is less than a page, a free fragment exists. Next time allocating bootmem, it will try to use the fragment if its size is enough. We may see that line 21, 23, 25, 27, and 31 shared the same page.

Line 22~23: Kernel tries to allocate bootmem for resource in resource_init(). The size of resource structure is 28 (0x1c) bytes.

Line 24~27: Kernel tries to allocate bootmem for saved_command_line and static_command_line in setup_command_line(), and both of them are of length 72 (0x49) bytes. The command line is also printed in line 29.

Line 28: Number of pages in this zone is 16384 (0x4000), and 128 of them are used for memory map. Therefore, the present_pages is 16384 - 128 = 16256.

Line 30~32: Kernel tries to allocate bootmem for PID hash table in pidhash_init(). pid_hash is 4 bytes, and the table is 256 * 4 = 1024 (0x400) bytes.

Line 33~38: Kernel tries to allocate bootmem for Dentry cache and Inode cache hash table in vfs_caches_init_early(). 1 hash list structure is 4 bytes. 8192 Dentry cache hash entries woul be 8192 * 4 = 0x8000 bytes. And 4096 Inode cache hash entries would be 0x4000 bytes.

Line 43~44: When the memory sub system is ready, it calls free_all_bootmem() that calls free_all_bootmem_core() for each bootmem bitmap to release all free pages to the buddy allocator. It examines the node_bootmem_map for all pages in the node (PFN 0 ~ 0x4000), calls __free_pages_bootmem() to release free either 32 pages or 1 page each time (depending on vec is all 1s or not). Free pages include PFN 0x5a9 ~ 0xfff and 0x108d ~ 0x3fff, and the total number of them is 0x39ca. And we can check the size of reserved memory: (0x4000 - 0x39ca) * 4096 = 6360K bytes. After freeing those pages, the pages storing bootmem bitmap can also be freed. It is 1 page in this example. Thus, the total number of released pages is 0x39cb (line 44).

Line 47: When kernel completes initial bootup and ready to start user-mode stuff, it calls free_initmem() to free .init section between symbol __init_begin and __init_end.

In conclusion,

available memory = total - reserved
MemTotal = total - reserved + .init
Bootmem allocator uses a simple bitmap representing the use of memory.
Reserved memory is mainly used to store kernel image.
The rest of reserved memory is for memory map, various hash tables, and some miscellaneous data.
After the memory sub system and zone allocator get ready, it will release the memory used for bootmem bitmap (bootmem retires).
When initial bootup is done, it will release the .init section that will never be used again.

Step by step to MIPS assembly

2010-06-21T02:29:00.000-07:00

In this article, I am going to introduce how to write MIPS assembly. All the examples introduced will able to be assembled or compiled into actual executable programs. They can be executed on a MIPS machine running Linux. So, let's forget the simulator and come to real world :-P. The first is the most simple program.

1         .data
2         .text
3         .global __start
4 __start:
5         li $4, 88
6         li $2, 4001
7         syscall

line 1: is an assembler directive which specifies that data section starts from here. It is empty since we don't have data section in this example.
line 2: specifies that text section starts from here.
line 3: specifies that __start is globally visible to all other modules that are part of the executable, and visible to the GNU linker. The symbol __start is required by the GNU Linker to specify the first instruction to be executed in a program, must always be a global one (same as _start for x86).

line 4: specifies the entry of __start
line 5: loads immediate value 88 to $a3
line 6: loads immediate value 4001 to $a0
line 7: calls the OS to perform system call.
line 5~7: executes the system call according to the number stored in $a3. For MIPS O32 ABI, the number starts from 4000, and exit() will be executed in this example. (You may reference linux kernel source include/asm-mips/unistd.h) The status code is stored in $a3, and we may check it by 'echo $?'.

Assemble the source code with the cross assembler.

$ mipsel-linux-uclibc-as -o exit1.o exit1.s

Link the object into an executable program.

$ mipsel-linux-uclibc-ld -o exit1 exit1.o

Then, test the program.

$ exit1
$ echo $?
88

OK. We have the skeleton of basic assembly source code. Normally, we don't write something like 'li $2, 4001', that is un-readable. It is easier to leverage definition of header files and the power of compiler. The same example calling exit() could be re-written.

 1 #include "asm/regdef.h"
 2 #include "asm/unistd.h"
 3 
 4         .data
 5         .text
 6         .global main
 7 main:
 8         .set noreorder
 9         .cpload        t9             # PIC ABI crap (function prologue)
10         .set reorder
11         li a0, 99
12         li v0, __NR_exit
13         syscall

line 1: we need register definitions
line 2: we need system call index definitions
line 6~7: is just like main() in a C program. Compiler will do some magic stuff (between __start and main) for our ease leveraging its power.
line 8~10: is the prologue of every function. Our program will compiled into a PIC, and linked with C library. We need the compiler to calculate a correct offset for global pointer register. It is a must when we want to make a call to the dynamic C library. We are going to have the same prologue in the following examples.

Compile the source code with cross compiler.

$ mipsel-linux-uclibc-gcc -o exit2 exit2.S

Test the program.

$ exit2
$ echo $?
99

Then, we may try to make some calls in our program. Instruction jal let us call a procedure. It will automatically place the return address in $ra (GPR 31), so that we can easily return back by 'jr ra'. The following example calculates the value of (2^3 + 5^2).

 1 # The illustration the use of function.
 2 # This program will compute the value of 2^3 + 5^2.
 3 
 4 #include "asm/regdef.h"
 5 #include "asm/unistd.h"
 6 
 7         .data
 8         .text
 9         .global main
10 main:
11         .set noreorder
12         .cpload        t9             # PIC ABI crap (function prologue)
13         .set reorder
14         subu sp, 4             # reserve 4bytes from stack
15         li a0, 2
16         li a1, 3
17         jal power              # calculate 2^3
18         nop
19         sw v0, 4(sp)           # push the value of 2^3
20         nop
21         li a0, 5
22         li a1, 2
23         jal power              # calculate 5^2
24         nop
25         lw a0, 4(sp)           # pop the value of 2^3
26         nop
27         addu sp, 4             # release 4bytes to stack
28         add a0, v0             # add with 5^2
29         li v0, __NR_exit
30         syscall
31 
32         .type power, @function
33 power:
34         add v0, a0, zero       # v0 = a0
35         subu a1, 1
36         beqz a1, ans           # power is 1
37         nop
38 ploop:
39         mul v0, v0, a0         # v0 * a0
40         subu a1, 1             # decrease the power
41         bnez a1, ploop         # next power
42         nop
43 ans:
44         jr ra
45         nop

line 14: reserves 4 bytes from stack that we are going to use.
line 15~16: passes argument to the function, power(). The first argument ($a0) is base, and the second argument ($a1) is exponent.
line 17: calls power().
line 18: is branch delay slot for jal
line 19~20: when power() returns, the value of 2^3 will be stored in $v0. We save is to the stack.
line 21~24: calls power() to calculate 5^2. It will be stored in $v0.
line 25~26: pop the stored value of 2^3 to $a0.
line 27: releases memory back to stack that we reserved.
line 28: adds $v0 (containing 5^2) with $a0 (containing 2^3).
line 29~30: calls exit(). The final answer is in $v0.
line 32: marks the symbol power as being a function name.
line 34: $v0 = $a0
line 35~37: if the exponent is equal to 1, the answer (in $v0) is equal to the base.
line 38~42: a loop to repeat calculation of multiplying base for (exponent - 1) times
line 43~45: returns to the original branch with final answer stored in $v0

Compile the source code with cross compiler.

$ mipsel-linux-uclibc-gcc -o power power.S

Check the value of 2^3 + 5^2.

$ power
$ echo $?
33

Note that the previous example is a simple usage of function. Is is recommended to follow the O32 ABI calling convention for a better compatibility.

With reference to previous example, you can write a hello-world program with write system call. There is also an example provided in Linux MIPS mailing list. How about a hello-world program calling C library printf? Take a look at the followed example.

 1 #include "asm/regdef.h"
 2 
 3         .globl main
 4 main:   .ent    main
 5         .frame sp, 32, ra
 6         .set   noreorder
 7         .cpload        t9
 8         .set   reorder
 9 
10         subu   sp, sp, 32
11         sw     ra, 28(sp)
12         .cprestore 24
13         la     a0, hello
14         jal    printf
15 
16         lw     ra, 28(sp)
17         addiu  sp, sp, 32
18         jr     ra
19         .end   main
20 
21         .data
22 hello:  .asciz  "Hello world\n"

line 4: marks the entry of main, the .ent pseudo-op is used by the debugger.
line 5: tells compiler frame register is $sp, frame size is 32 bytes, and return register is $ra.
line 6~8: is the prologue described in previous example.
line 10: reserves the stack memory for 32 bytes. That includes space for local variables, saved registers, and argument area. Please reference the calling convention article for detail.

line 11: pushes the return address. When we call other functions, return address will be changed. Thus, it must be saved first.
line 12: restores the global pointer back. Because we will make a call that clobbers $gp, we have to restore the $gp value before returning. The idea is similar to line 11 that saving $ra.

line 13: the argument is string "Hello world"
line 14: make a call to printf
line 16: restores back the return address
line 17: release the stack that we reserved
line 18: returns from main function
line 19: marks the end of main
line 22: the "Hello world" string is stored in data section

Compile the source code with cross compiler.

$ mipsel-linux-uclibc-gcc -o printf printf.S

Test the hello-world program.

$ printf
Hello world

Having these examples for reference, I think it will be easier if you'd like to write assembly together with C programs.

References:

MIPS calling convention with GCC

2009-12-04T08:04:00.000-08:00

In this article, we are going to learn MIPS calling conventions by examples. And we will focus on the one used by GCC. All the examples will be compiled with GCC flag -mno-abicalls not generating SVR4-style position-independent code. Normally -mabicalls should be used, we disable it for the simplification of assembly codes.

First up, take a look at the most simple leaf function without any arguments.

int empty(void)
{
return 0;
}

 1 empty:
 2         addiu  $sp,$sp,-8
 3         sw     $fp,0($sp)
 4         move   $fp,$sp
 5         move   $2,$0
 6         move   $sp,$fp
 7         lw     $fp,0($sp)
 8         addiu  $sp,$sp,8
 9         j      $31
10         nop

line 2: reserve 8 bytes stack space
line 3: push the previous $fp to stack
line 4: take current $sp as the subroutine's frame pointer
line 5: set $v0 to zero ($v0 stores the return value)
line 6,7,8: pop out $fp from stack, restore back original $sp
line 9: jump to return address
line 10: branch delay slot

The function has its own stack frame. It must save the old frame pointer ($fp) before using it, and restore it back before returning to the caller. Generally speaking, if any of $16..$23 or $29..$31 is changed within the called function, it must be saved in the stack frame before use and restored from the stack frame before return from the function. It is called general register save area. As to the example, 4 bytes is enough for saving $fp, why does it reserve 8 bytes stack space? It is because that the register save area must be doubleword (8 byte) aligned.

Besides saving $fp, a non-leaf function must save return address ($ra) as well. When it calls other subroutines, the $ra will be modified for returning from the subroutines. The non-leaf function has to save $ra in order to return to its caller.

int emptycaller(void)
{
empty();
return 0;
}

 1 emptycaller:
 2         addiu  $sp,$sp,-24
 3         sw     $31,20($sp)
 4         sw     $fp,16($sp)
 5         move   $fp,$sp
 6         jal    empty
 7         nop
 8         move   $2,$0
 9         move   $sp,$fp
10         lw     $31,20($sp)
11         lw     $fp,16($sp)
12         addiu  $sp,$sp,24
13         j      $31
14         nop

line 3: push $ra onto the stack
line 4: push $fp onto the stack
line 10: pop $ra from the stack
line 11: pop $fp from the stack
In the example, 8 bytes stack space should be enough for saving $ra and $fp. The additional 16 bytes (total 24 bytes) is the function call arguemnt area, and we will dicuss about it later.

We know that arguments are passed to subroutines in argument registers ($a0, $a1, $a2, and $a3). Let's take a look at the simple leaf function with 3 arguments.

int args3(int x, int y, int z)
{
return (x + y + z);
}

 1 args3:
 2         addiu  $sp,$sp,-8
 3         sw     $fp,0($sp)
 4         move   $fp,$sp
 5         sw     $4,8($fp)
 6         sw     $5,12($fp)
 7         sw     $6,16($fp)
 8         lw     $3,8($fp)
 9         lw     $2,12($fp)
10         nop
11         addu   $2,$3,$2
12         lw     $3,16($fp)
13         nop
14         addu   $2,$2,$3
15         move   $sp,$fp
16         lw     $fp,0($sp)
17         addiu  $sp,$sp,8
18         j      $31
19         nop

line 2~4: the function prologue as described previously
line 5~7: push the arguments from $a0, $a1, and $a2 to the stack
line 8~14: add the 3 arguments, and put the sum in $v0 for the return value
line 15~17: the function epilogue as described previously
line 18~19: return to the caller
The arguments are pushed to 8($fp) to 16($fp), which are located in the caller's stack frame. A non-leaf function (Caller) will always reserve the top 4 words (16 bytes) at least to storing arguments to called functions. It is called the function call argument area.

If the maximum number of arguments to the called function is fewer than 4 words, the caller still has to reserve 4 words. Take the followed non-leaf function for example. The function reserved 24 bytes stack space instead of 8, and the additional 16 bytes is the function call argument area.

int arg3caller(void)
{
args3(5, 6, 7);
return 0;
}

 1 arg3caller:
 2         addiu  $sp,$sp,-24
 3         sw     $31,20($sp)
 4         sw     $fp,16($sp)
 5         move   $fp,$sp
 6         li     $4,5                        # 0x5
 7         li     $5,6                        # 0x6
 8         li     $6,7                        # 0x7
 9         jal    args3
10         nop
11         move   $2,$0
12         move   $sp,$fp
13         lw     $31,20($sp)
14         lw     $fp,16($sp)
15         addiu  $sp,$sp,24
16         j      $31
17         nop

line 2: reserve 24 bytes stack space
line 3~4: Since 0($sp) ~ 12($sp) is function call argument area, the old $fp will be pushed onto 16($sp), and $ra will be pushed onto 20($sp).
line 6~10: pass the arguments in $a0, $a1, and $a2

There are four argument registers ($a0, $a1, $a2, and $a3) totally. When calling a function with more than 4 arguemnts, the first 4 arguments are passed in registers, and the others are passed on the stack.

int args6(int x, int y, int z, int a, int b, int c)
{
return (x + y + z + a + b + c);
}

 1 args6:
 2         addiu  $sp,$sp,-8
 3         sw     $fp,0($sp)
 4         move   $fp,$sp
 5         sw     $4,8($fp)
 6         sw     $5,12($fp)
 7         sw     $6,16($fp)
 8         sw     $7,20($fp)
 9         lw     $3,8($fp)
10         lw     $2,12($fp)
11         nop
12         addu   $2,$3,$2
13         lw     $3,16($fp)
14         nop
15         addu   $2,$2,$3
16         lw     $3,20($fp)
17         nop
18         addu   $2,$2,$3
19         lw     $3,24($fp)
20         nop
21         addu   $2,$2,$3
22         lw     $3,28($fp)
23         nop
24         addu   $2,$2,$3
25         move   $sp,$fp
26         lw     $fp,0($sp)
27         addiu  $sp,$sp,8
28         j      $31
29         nop

line 2~4: function prologue
lin 5~8: push the first 4 arguments to the stack
line 9~18: add the first 4 arguments, and put the sum in $v0
line 19~21: fetch the 5th argument from 24($fp), and add it to $v0
line 22~24: fetch the 6th argument from 28($fp), and add it to $v0 for the return value
line 25~27: function epilogue
line 28~29: return to the caller

In order to call the 6-argument function, the caller has to push the 5th and 6th arguments onto the stack. The function call argument area becomes of length 6 words (24 bytes) instead of 4 words. It reserves total 32 bytes, 8 bytes for registers $ra and $fp, and 24 bytes for function call argument area. Note that the area will also be doubleword aligned.

int arg6caller(void)
{
args5(5, 6, 7, 8, 9, 10);
return 0;
}

 1 arg6caller:
 2         addiu  $sp,$sp,-32
 3         sw     $31,28($sp)
 4         sw     $fp,24($sp)
 5         move   $fp,$sp
 6         li     $2,9                        # 0x9
 7         sw     $2,16($sp)
 8         li     $2,10                       # 0xa
 9         sw     $2,20($sp)
10         li     $4,5                        # 0x5
11         li     $5,6                        # 0x6
12         li     $6,7                        # 0x7
13         li     $7,8                        # 0x8
14         jal    args5
15         nop
16         move   $2,$0
17         move   $sp,$fp
18         lw     $31,28($sp)
19         lw     $fp,24($sp)
20         addiu  $sp,$sp,32
21         j      $31
22         nop

line 2: reserve 32 bytes stack space
line 3~4: Since 0($sp) ~ 20($sp) is function call argument area, the old $fp will be pushed onto 16($sp), and $ra will be pushed onto 20($sp).
line 6~9: pass the 5th and 6th arguments on the stack
line 10~13: pass the first 4 arguments in argument registers

Next, we are going to see a little bit complicated example. We declare 2 variables stored in registers $s0 and $s1. The function must save them in the general register save area before using them. Besides, we declare 2 local variables (automatic variables), and they will also be saved on the top of the general register save area.

int comp(int m, int n)
{
int a = 1, b = 2;
register int x = 5, y = 6;
args6(a, b, m, n, x, y);
return (x + y);
}

Therefore, the stack frame will be like:

The assembly code line by line:

 1 comp:
 2         addiu  $sp,$sp,-48
 3         sw     $31,44($sp)
 4         sw     $fp,40($sp)
 5         sw     $17,36($sp)
 6         sw     $16,32($sp)
 7         move   $fp,$sp
 8         sw     $4,48($fp)
 9         sw     $5,52($fp)
10         li     $2,1                        # 0x1
11         sw     $2,24($fp)
12         li     $2,2                        # 0x2
13         sw     $2,28($fp)
14         li     $16,5                       # 0x5
15         li     $17,6                       # 0x6
16         sw     $16,16($sp)
17         sw     $17,20($sp)
18         lw     $4,24($fp)
19         lw     $5,28($fp)
20         lw     $6,48($fp)
21         lw     $7,52($fp)
22         jal    args6
23         nop
24         addu   $2,$16,$17
25         move   $sp,$fp
26         lw     $31,44($sp)
27         lw     $fp,40($sp)
28         lw     $17,36($sp)
29         lw     $16,32($sp)
30         addiu  $sp,$sp,48
31         j      $31
32         nop

line 3~4: save previous $ra and $fp
line 5~6: save $s0 and $s1 before using it
line 8~9: save $a0 and $a1 (arguments m and n)
line 10~13: storing local variable a and b
line 14~15: variable x and y are saved in registers $s0 and $s1
line 16~17: pass arguments x and y on stack
line 18~21: pass arguments a, b, m, and n in argument registers
line 22~23: call the subroutine
line 24: put the sum of x + y to $v0 for the return value
line 26~30: restore registers back
line 31~32: return to the caller

References:

C99 Features

2009-11-19T01:25:00.000-08:00

Here is a simple example demonstrating some of the C99 features. It is a test program I wrote for learning from C99 standard. Although it doesn't cover all the new features for C99, I think it is at least a start for people who are not familiar to C99 like me.

  1 #include <stdio.h>
  2 #include <stdlib.h>
  3 #include <complex.h>
  4 
  5 #define ppp printf("\n%s, line %d\n", __func__, __LINE__)
  6

line 3: include the header for complex definations and declarations

  7 void data_type(void)
  8 {
  9     _Bool b = 2;
 10     long long int l;
 11     _Complex c = 1 + 2.3 * I;
 12     ppp;
 13 
 14     printf("  b = %d, sizeof b = %d\n", b, sizeof(b));
 15     printf("  sizeof l = %d\n", sizeof(l));
 16     printf("  c = %f + %f i, sizeof c = %d\n", creal(c), cimag(c), sizeof(c));
 17 }
 18

line 9: boolean type
line 10: long long type
line 11: comlex type
line 15: the value of a boolean variable can only be 0 or 1
line 16: print the real number and imaginary number of the complex number

 19 void variable_length_array(int n)
 20 {
 21     int arr[n];
 22     ppp;
 23 
 24     printf("  arr: ");
 25     for (int i = 0; i < n; i++) {
 26         arr[i] = i;
 27         printf("%d ", arr[i]);
 28     }
 29     printf("\n");
 30 
 31     int arr2[n][n];
 32     printf("  size of arr2 = %d\n", sizeof(arr2)/sizeof(int));
 33 }
 34

line 21: the lenght of the array is determined at run time
line 24: c++ style for loop initialization
line 31: two dimention array is fine, too

 35 #define dprint(...) printf(__VA_ARGS__)
 36 void variadic_macro(void)
 37 {
 38     ppp;
 39     dprint("  hello!\n");
 40     dprint("  %d + %d = %d\n", 2, 1, 3);
 41 }
 42

line 35: __VA_ARGS__ will be replaced by the passed arguments during macro expansion
line 39, 40: variable arguemnts are used for the macro

 43 struct abc { int a; int b; int c[10]; };
 44 int foo(void) { /* do something... */ return 0; }
 45

 46 void designated_initializer(void)
 47 {
 48     ppp;
 49 
 50     struct abc s1 = { .b = 1, .a = -1 };
 51     struct abc s2 = { .c[2] = 2 };
 52     printf("  s1.a = %d, s1.b = %d, s2.c[2] = %d\n", s1.a, s1.b, s2.c[2]);
 53 
 54     int arr[10] = { [2] = 22, [5] = 55 };
 55     struct abc s3[] = { [3].c[2] = 33 };
 56     printf("  arr[5] = %d, s3[3].c[2] = %d\n", arr[5], s3[3].c[2]);
 57 
 58     struct abc s4 = { .b = 4, .b = 5, .b = 6 };
 59     printf("  s4.b = %d\n", s4.b);
 60 
 61     struct xx { int x; struct abc sub; };
 62     struct xx s5 = { .sub.b = 5 };
 63     struct xx s6 = { .sub = s1 };
 64     printf("  s5.sub.b = %d, s6.sub.b = %d\n", s5.sub.b, s6.sub.b);
 65 }
 66

line 50: members can be named in initializer lists
line 51: initialize only the 3rd object for the array member c
line 54, 55: initialize some array objects only
line 58: initialize several times, and the last one wins
line 62: initialize member b of the sub structure
line 63: initialize only the sub structure

 67 void cl(struct abc s)
 68 {
 69     printf("  s.a = %d, s.b = %d\n", s.a, s.b);
 70 }
 71 void compound_literal(void)
 72 {
 73     int *p = (int []){1, 2, 3};
 74     ppp;
 75 
 76     printf("  *p = %d, *(p+1) = %d\n", *p, *(p+1));
 77     cl((struct abc){ .a=9, .b=8 });
 78 }
 79

line 73: composed with a construct with a brace'd initializer to create unnamed array's.
line 77: pass a compound literal argument to function cl (both compound literal and designated initializer are used)

 80 void flexible_array_member(void)
 81 {
 82     struct xx { int a; int arr[]; };
 83     struct xx *p;
 84 
 85     p = malloc(10 * sizeof(int));
 86     p->arr[9] = 9;
 87     printf("  p->arr[9] = %d\n", p->arr[9]);
 88     free(p);
 89 
 90     p = malloc(20 * sizeof(int));
 91     p->arr[19] = 19;
 92     printf("  p->arr[19] = %d\n", p->arr[19]);
 93     free(p);
 94 }
 95

line 82: incomplete array without bounds of struct members is allowed (it must be the last member)
line 85: set p to a struct xx with array of length 9
line 90: set p to another struct xx with array of length 19

 96 int main(void)
 97 {
 98     data_type();
 99     variable_length_array(5);
100     variadic_macro();
101     designated_initializer();
102     compound_literal();
103     flexible_array_member();
104 
105     //return 0;
106 }

line 105: compiler will add a 'return 0;' implicitly

The result of this program:

$ gcc -Wall -std=c99 c99.c
$ ./a.out

data_type, line 12
b = 1, sizeof b = 1
sizeof l = 8
c = 1.000000 + 2.300000 i, sizeof c = 16

variable_length_array, line 22
arr: 0 1 2 3 4
size of arr2 = 25

variadic_macro, line 38
hello!
2 + 1 = 3

designated_initializer, line 48
s1.a = -1, s1.b = 1, s2.c[2] = 2
arr[5] = 55, s3[3].c[2] = 33
s4.b = 6
s5.sub.b = 5, s6.sub.b = 1

compound_literal, line 74
*p = 1, *(p+1) = 2
s.a = 9, s.b = 8
p->arr[9] = 9
p->arr[19] = 19

References:

Learning x86 Calling Conventions by Examples

2009-11-16T03:36:00.000-08:00

The x86 architecture features many different calling conventions. There are three major calling conventions in use: __cdecl, __stdcall, and __fastcall. We are going to force the compiler to generate assembly with different calling conventions in order to see how they work.

Here is a simple function in C:

int f(int a, int b)
{
return a + b;
}

GCC compiler uses __cdecl by default (It is also the default calling convention for Microsoft C/C++ compiler). We will get the same result as __cdecl without extra compiler options or configurations.

Let's call the simple function with __cdecl calling convention:

int __attribute__((cdecl)) f(int a, int b)
{
return a + b;
}
void caller(void)
{
f(3, 5);
}

The assembly generated by gcc for __cdecl would be:

 1 f:
 2         pushl  %ebp
 3         movl   %esp, %ebp
 4         movl   12(%ebp), %eax
 5         addl   8(%ebp), %eax
 6         popl   %ebp
 7         ret
 8 caller:
 9         pushl  %ebp
10         movl   %esp, %ebp
11         subl   $8, %esp
12         movl   $5, 4(%esp)
13         movl   $3, (%esp)
14         call   f
15         leave
16         ret

Calling a __cdecl function line by line:

Push parameters onto the stack, from right to left. In this example, 5 first and then 3.

12         movl   $5, 4(%esp)
13         movl   $3, (%esp)

Call the function f. The process will push the content of %eip onto the stack (for the return address of function f).

14         call   f

Since we are in the new function f, a new local stack frame is needed. We push the current %ebp (which belongs to the caller's frame) onto the stack, and make it point to the top of the stack (%esp).

 2         pushl  %ebp
 3         movl   %esp, %ebp

After %ebp is changed, the arguments of the function f will be referred as 8(%ebp) and 12(%ebp). They are summed to %eax, which will be the return value to the caller. Note that 0(%ebp) is the old (caller's) base pointer, and 4(%ebp) is the old instruction pointer.

 4         movl   12(%ebp), %eax
 5         addl   8(%ebp), %eax

Restore the saved base pointer %ebp.

 6         popl   %ebp

Return from the function. It will pop the old %eip and jump to the location.

 7         ret

Note that line 9, 10 and line 2, 3 pairs are the same. It is the function prologue used for every function. Line 8 reserves space for arguments of function f. Line 9 to 11 together is actually the same as ENTER instruction. The compiler uses 3 instructions instead of ENTER instruction for performance.

 9         pushl  %ebp
10         movl   %esp, %ebp
11         subl   $8, %esp

LEAVE instruction will set %esp to %ebp, then pop %ebp. It is the epilogue used for every function. Since the stack is intact in function f, we don't have to set %esp to %ebp, popping %ebp is enough. Compiler uses LEAVE instead of 2 instructions also for performance.

15         leave

As we can see, main characteristics of __cdecl calling convention are:

Arguments are passed from right to left, and placed on the stack.
Stack cleanup is performed by the caller.

How about calling the function with __stdcall?

int __attribute__((stdcall)) f(int a, int b)
{
return a + b;
}
void test(void)
{
f(3, 5);
}

The assembly for __stdcall would be:

 1 f:
 2         pushl  %ebp
 3         movl   %esp, %ebp
 4         movl   12(%ebp), %eax
 5         addl   8(%ebp), %eax
 6         popl   %ebp
 7         ret    $8
 8 test:
 9         pushl  %ebp
10         movl   %esp, %ebp
11         subl   $8, %esp
12         movl   $5, 4(%esp)
13         movl   $3, (%esp)
14         call   f
15         subl   $8, %esp
16         leave
17         ret

The only difference from __cdecl is that it uses "ret 8" (line 7) for self clean up stack. Therefore, the caller needs to "subl 8, %esp" (line 15) in order to retrieve %ebp back before using leave instruction.

As we can see, main characteristics of __stdcall calling convention are:

Arguments are passed from right to left, and placed on the stack.
Stack cleanup is performed by the called function.

And how about calling the function with __fastcall?

int __attribute__((fastcall)) f(int a, int b)
{
return a + b;
}
void test(void)
{
f(3, 5);
}

The assembly generated by gcc for __fastcall would be:

 1 f:
 2         pushl  %ebp
 3         movl   %esp, %ebp
 4         subl   $8, %esp
 5         movl   %ecx, -4(%ebp)
 6         movl   %edx, -8(%ebp)
 7         movl   -8(%ebp), %eax
 8         addl   -4(%ebp), %eax
 9         leave
10         ret
11 test:
12         pushl  %ebp
13         movl   %esp, %ebp
14         movl   $5, %edx
15         movl   $3, %ecx
16         call   f
17         popl   %ebp
18         ret

It indicates that the arguments should be placed in registers, rather than on the stack, whenever possible. The argument first argument 3 is placed in %ecx (line 15) and the second argument 5 is in %edx (line 14). Function f copy the arguments to its own stack, and calculate with the stack.

What if calling a __fastcall function more thatn two argument?

int __attribute__((fastcall)) f(int a, int b, int c)
{
return a + b + c;
}
void test(void)
{
f(3, 5, 7);
}

 1 f:
 2         pushl  %ebp
 3         movl   %esp, %ebp
 4         subl   $8, %esp
 5         movl   %ecx, -4(%ebp)
 6         movl   %edx, -8(%ebp)
 7         movl   -8(%ebp), %eax
 8         addl   -4(%ebp), %eax
 9         addl   8(%ebp), %eax
10         leave
11         ret    $4
12 test:
13         pushl  %ebp
14         movl   %esp, %ebp
15         subl   $4, %esp
16         movl   $7, (%esp)
17         movl   $5, %edx
18         movl   $3, %ecx
19         call   f
20         subl   $4, %esp
21         leave
22         ret

The third argument 7 is pushed onto stack (line 16). And function f uses "ret 4" for cleaning up the stack. Therefore, we may conclude that main characteristics of __stdcall calling convention are:

The first two function arguments that require 32 bits or less are placed into registers ECX and EDX.
The rest of them are pushed on the stack from right to left.
If any stack based arguments were present, the callee cleans them off of the stack

These simple examples are just for providing a basic idea of how these 3 calling convention work. For more detail, the followed links are suggested to read.

References:

Understand Weak Symbols by Examples

2009-11-08T23:08:00.000-08:00

Wikipedia defines the weak symbols: "In computing, a weak symbol is a symbol definition in an object file or dynamic library that may be overridden by other symbol definitions, its value will be zero if no definition found by loader." In other words, we can define a symbol that doesn't need to be resolved at link time. It is a very well-known feature and used a lot in Linux Kernel, Glibc, and so on.

Take a look at the example, we are not able to compile it due to the 'undefined reference' error.

$ cat err.c
int main(void)
{
        f();
        return 0;
}

$ gcc err.c
/tmp/ccYx7WNg.o: In function `main':
err.c:(.text+0x12): undefined reference to `f'
collect2: ld returned 1 exit status

Try to declare 'f' as an weak symbol, and we can compile it without error.

$ cat weak.c
void __attribute__((weak)) f();
int main(void)
{
        if (f)
        f();
        return 0;
}
$ gcc weak.c

Note that the function 'f' is called inside an if statement. If not calling 'f' this way, we will get a 'Segmentation fault' error. In the weak.c example, 'f' is actually not invoked. It is because 'f' is an un-defined weak symbol and therefore will be zero when the loader cannot find it.

$ ./a.out
$ nm a.out
...
w f
08048324 T main
...

Let's define the function 'f' in another file, and link the objects together. This time, 'f' will be correctly called. (Note that puts is the optimization of printf by gcc)

$ cat f.c
#include <stdio.h>
void f(void)
{
        printf("hello from f\n");
}

$ gcc -c weak.c f.c
$ gcc -o weak weak.o f.o
$ ./weak
hello from f

$ nm weak.o
w f
00000000 T main
$ nm f.o
00000000 T f
U puts
$ nm weak
...
08048384 T f
08048354 T main
U puts@@GLIBC_2.0
...

We may even override the original weak symbol (type 'W') with a strong symbol (type 'T').

$ cat orig.c
#include <stdio.h>
void __attribute__((weak)) f()
{
        printf("original f..\n");
}
int main(void)
{
        f();
        return 0;
}
$ gcc orig.c
$ ./a.out
original f..


$ cat ovrd.c
#include <stdio.h>
void f(void)
{
        printf("overridden f!\n");
}
$ gcc -c orig.c ovrd.c
$ gcc -o ovrd orig.o ovrd.o
$ ./ovrd
overridden f!


$ nm orig.o
00000000 W f
00000014 T main
U puts
$ nm ovrd.o
00000000 T f
U puts
$ nm ovrd
...
0804838c T f
08048368 T main
U puts@@GLIBC_2.0
...

And of course, we can also override a weak object (type 'V') with a strong object (type 'D').

$ cat orig-obj.c
#include <stdio.h>
int __attribute__((weak)) x = 1;
int __attribute__((weak)) y = 1;
int main(void)
{
        printf("x = %d, y = %d\n", x, y);
        return 0;
}
$ gcc orig-obj.c
$ ./a.out
x = 1, y = 1


$ cat ovrd-obj.c
int x = 2;
void f(void)
{
}
$ gcc -c orig-obj.c ovrd-obj.c
$ gcc -o ovrd-obj orig-obj.o ovrd-obj.o
$ ./ovrd-obj
x = 2, y = 1


$ nm orig-obj.o
00000000 T main
U printf
00000000 V x
00000004 V y
$ nm ovrd-obj.o
00000000 T f
00000000 D x
$ nm ovrd-obj
...
08048394 T f
08048354 T main
U printf@@GLIBC_2.0
080495c8 D x
080495c4 V y
...

What if there are multiple symbols? Linker's symbol rules tell us that:

Multiple strong symbols are not allowed
Given a strong symbol and multiple weak symbols --> choose the strong symbol
Given multiple weak symbols --> choose any of those weak symbols

$ cat mul.c
int main(void)
{
        f();
        return 0;
}
$ cat s1.c
#include <stdio.h>
void f(void)
{
        printf("1st strong f from %s\n", __FILE__);
}
$ cat s2.c
#include <stdio.h>
void f(void)
{
        printf("2nd strong f from %s\n", __FILE__);
}
$ cat w1.c
#include <stdio.h>
void __attribute__((weak)) f(void)
{
        printf("1st weak f from %s\n", __FILE__);
}
$ cat w2.c
#include <stdio.h>
void __attribute__((weak)) f(void)
{
        printf("2nd weak f from %s\n", __FILE__);
}
$ gcc -c mul.c s1.c s2.c w1.c w2.c


$ gcc -o test1 mul.o s1.o s2.o
s2.o: In function `f':
s2.c:(.text+0x0): multiple definition of `f'
s1.o:s1.c:(.text+0x0): first defined here
collect2: ld returned 1 exit status


$ gcc -o test2 mul.o s1.o w1.o w2.o
$ ./test2
1st strong f from s1.c


$ gcc -o test3-1 mul.o w1.o w2.o
$ ./test3-1
1st weak f from w1.c
$ gcc -o test3-2 mul.o w2.o w1.o
$ ./test3-2
2nd weak f from w2.c

Hope these examples help!

References:

Debian Backporting

2008-08-21T02:02:00.000-07:00

It is easy to install packages on Debian. All you have to do is "apt-get install package". But what if we would like to install a newer version package? For example, git-core version on Debian stable is 1.4.4. It is too old and does not have basic commands such as git config, init,... etc. We may install git from Debian Backports. It is a easier way than building from source for most people.

add the backport mirrors to /etc/apt/sources.list
deb http://ftp.tw.debian.org/backports.org/ etch-backports main non-free
deb-src http://ftp.tw.debian.org/backports.org/ etch-backports main non-free
import backports.org archive's key into apt
# apt-get install debian-backports-keyring
(or # wget -O - http://backports.org/debian/archive.key | apt-key add -)
resynchronize the package index files from their sources
# apt-get update
remove previously installed git
# apt-get remove git-core
install git from
# apt-get -t etch-backports install git-core
(or # aptitude -t etch-backports install "git-core")
finish
# git --version
git version 1.5.6.3

We have git 1.5.6.3 installed. Then, happily coding. :P
For more information, take a look at Debian Backports wiki.

Remove allow_branch_to_undefined from U-boot

2008-08-20T02:53:00.000-07:00

Old U-boot (v1.1.3 in my example, too old hum?) needs a option -mips_allow_branch_to_undefined for ELDK AS to assemble. We have to modify cpu/mips/start.S so that AS of other toolchains could assemble it without an error - Error: can't resolve `_GLOBAL_OFFSET_TABLE_' {*UND* section} - `L16' {.text section}

original start.S:

/* Initialize GOT pointer.
*/
bal     1f
nop
.word   _GLOBAL_OFFSET_TABLE_ - 1f + 4
1:
move    gp, ra
lw      t1, 0(ra)
add     gp, t1

It counts an offset first, and gets the gp later from the return address and the offset. Let us do it with different method.

new start.S:

bal     1f
nop
.word   _GLOBAL_OFFSET_TABLE_
1:
lw      gp, 0(ra)

Second, remove -Wa,-mips_allow_branch_to_undefined from cpu/mips/config.mk. Then we can make Uboot successfully with the toolchain built from buildroot.

交車需知大全

2008-08-11T17:30:00.000-07:00

前陣子買車前到處爬文整理起來的交車需知大全，本來一直放在googole docs沒公開，剛看到有直接張貼到blogger的功能，拿來試用看看，有興趣的人可以拿來參考... :)

一、基本需知

交車一定要選在大白天，晚上會看不清楚車輛烤漆和內裝的瑕疵
不選在下午四、五點交車，因為員工趕著下班。

二、證件核對

詳細核對契約書內容，業代贈品是否符合。點交所有車籍證件是否正確齊全，稅單發票等等都不可少

檢驗合格證：

該份資料為商檢局做汽車品質的測試證明單。

貨物稅完稅證明、發票、牌照稅及燃料稅單：

這些稅單代表新出廠的車輛完全沒有問題，可上路，是否納清，請核對。

出廠證明書、海關進口證明書：

證明書內明有生產日期、汽車年份、生產地，而且有海關進口書，其中會標示產地國、生產日期....等資料，為汽車進口重要資料。

牌照申請書：

該份資料是日後所有權轉移，變更地址申請等車籍原始資料，到監理處所要用，要保管好。如發生遺失，馬上到監理所申請補發。不然車子被過戶也不知道。

行照及保險卡：

行照內記載有牌照號碼、型式、顏色、引擎及車身號碼等資料，要確認該份資料與車是否相符，
核對行照車主，地址是否有錯誤。
原發照日期和換補照日期是否相同、有效日期(應為三年)、指定檢驗日期(應為五年)
保險卡的核對工作主要有車主、車號、車子種類。保險種類、自付額多少等資料，與當初承保條件有無相符

車籍資料袋：

每輛車的車籍資料袋內另附說明書、保養手冊說明保養須知，各地服務廠商地點、電話。
尤其是保證書部份，要看清楚保證內容是否公平，其範圍為何，保證幾年幾萬公里，不在保證之列的零件就要小心檢查狀況。
記得是向那位業務員買車，有問題馬上通知業務員，切忌懷有省事事省的駝鳥心態，等到小毛病變大毛病
車主手冊、保養手冊(有無經銷商蓋章)、點煙器(這個通常會隨車主手冊一起領取)

三、車況檢查

引擎室

引擎蓋

與二邊前葉子鈑的間隙是否平均，如果沒有可以請技師當場調整。

開啟和關閉是否容易操作，順便請技師幫新手介紹一下簡易保養的重點部位，並瞭解一下車身號碼和引擎號碼的標示處。

油箱蓋

開啟和關閉是否容易操作，免得萬一因為沒辦法開啟加油叨在路上，那會粉臭哩
當然也可以請業代事先在交車前幫泥加滿油(可對發票來核準)

檢查五油三水是否充足，有無漏油(水)，冷卻水是否有添加防鏽劑
檢查各油、水箱的開關及錶狀況是否良好
再檢查各管路導線是否固定在定位，引擎皮帶(充電馬達同)風扇皮帶作用是否良好

五油

機油
機油量標準量法：將車停放平坦路面，拉起手煞車，關掉引擎，並等待數分鐘，讓機油流回油底殼。取出油尺，並用清潔的布拭乾淨。插回油尺，且要完全放到底。再次取出油尺，正常的油位高度必須在油尺上的Ｍａｘ與Ｍｉｎ兩個記號之間。
變速箱油
自排油量標準量法：發動引擎，讓車行駛，使水溫表指針達到正常工作溫度；將車停放平坦路面，拉起手煞車（不可關掉引擎）。引擎繼續怠速運轉，腳踩煞車踏板；將排檔桿從Ｐ檔位逐一換至Ｌ檔位，並在各檔位停留數秒鐘。最後將排檔桿排在P檔位。取出油尺，並用清潔的布拭乾淨。插回油尺，且要完全放到底。再次取出油尺，正常的油位高度必須在油尺上HOT範圍內。
動力方向機油
動力方向機油量標準量法：將車停放平坦路面，拉起手煞車，取出油尺，並用清潔的布拭乾淨。插回油尺，且要完全放到底。再次取出油尺，正常的油位高度必須在油尺上HOT範圍內。
煞車油
煞車油是最重要的一種油，依照規定通常一年或三萬公里更換一次（以先到者為準），若還未到更換時機，也要檢視煞車油存量，您可先由煞車油壺外的刻度判別，煞車油必須介於Ｍａｘ與Ｍｉｎ之間，若有減少一定要添加至Ｍax的位置，而號數不同的煞車油更是不可混用。
汽油
行車上路前一定要檢查汽油量是否充足。

四壓

油壓
機油壓力指示燈因在鎖匙開紅火時亮(未亮請進廠做進一步的檢修。)，引擎發動後十秒內熄滅，如未在十秒內熄滅請馬上熄火，檢查機油量。如充足請請進廠做進一步的檢修。(若在行駛途中亮起時，請馬上找地方停下來，看一下油尺，檢查機油量是否足夠，不夠時一定要加足。)
電壓
電瓶指示燈因在鎖匙開紅火時亮(未亮請進廠做進一步的檢修。)，引擎發動後十秒內熄滅，如未在十秒內熄滅請進廠做進一步的檢修。
胎壓
每日行車前應檢查全部輪胎胎壓是否正常(一般車輛在車門柱上或手套箱內有胎壓指示表)
氣壓(真空輔助)
將煞車踏板踏死後起動引擎看踏板踏是否會下沉，若煞車踏板會下沉即表示真空輔助煞車作用正常，若沒下沉車主可將煞車總泵輔助器上有一條從進氣岐管接至煞車總泵輔助器的黑色軟管檢查是否有無真空作用，老舊破裂，阻塞，若沒有以上的徵狀請進廠做進一步的檢修。

三水

水箱水
水箱中的液體為水與水箱精，平時應常檢查水箱及副水箱內的儲水量是否足夠，如果副水箱的水位有降低，應在冷車時以清水或水箱精加至上限與下限的中間即可，若要自我檢視水箱的儲水量，請不要在引擎運轉時或停止運轉一小時內打開水箱?#092;，以免因高溫的水箱水噴出而被燙傷。
電瓶水
電瓶水的部份，一樣有注水的上、下限，不足時請以純水或電瓶水填加，切勿添加自來水。
雨刷水
雨刷水也是需要經常檢查水位是否足夠，雨刷水最好能添加一些能夠去除擋風玻璃上的油漬之清潔液，以普通的清水代替也可，雨刷水不足雖然不會讓車輛故障，但擋風玻璃被前車噴到的髒污無法及時清除，也是有可能會影響行車安全。

外觀

車燈

檢查所以燈泡是否都會亮，玻璃是否有破損，燈殼是否有破損或水氣。
前方：方向燈、前小燈、大燈、、遠光燈閃動、遠光燈固定、暫停雙黃閃燈、霧燈測試
車側：側方向燈
後方：方向燈、後小燈、煞車燈、暫停雙黃閃燈、後霧燈(如果有的話)、第三煞車燈、倒車燈測試試
若可以，試一下大燈照出去的位置是否太高或太低

輪胎

輪胎胎壓正常與否。

車門

電動後視鏡，折疊或調整是否正常。
試開關每一個車門是否開關順暢，關上時有無異音。
測試後行李箱蓋開啟和關閉是否容易操作，並檢查與車體的間隙是否平均正常。
檢查車門膠條等是否安裝妥當、車門與車體接縫處是否均勻、車門邊的板金烤漆是否均勻。

車窗

電動窗及天窗開啟上掀等動作是否上下正常，有無異音產生。
玻璃與窗框接縫是否密合
前後雨刷的各項測試如間歇雨刷功能、正常速及快速雨刷功能、噴水測試，有後檔雨刷配置的車款，也要做後雨刷功能測試。

車身

站在離車身3公尺遠的地方，檢查車身烤漆顏色是否有色差，觀察車輛塗裝(噴漆)是否良好
版件組裝是否平順，然後貼近車身，看烤漆是否平順或刮傷，有無流汗或顆粒(可用手撫摸)
車身飾條邊及裝潢部份是否完整，有沒有劃破銬漆屑，接縫處是否生鏽起泡。

檢查車底是否有生鏽
有無組裝產生的傷痕或割痕，尤其是你有外加配件，小心技師不小心傷到烤漆保桿等等。

如果是有買天窗的車款，可做天窗漏水測試(時間建議至少數分鐘，可以由人員用水管不停的灑水於天窗上測試之)。

內裝

組裝：

門板是否牢靠，方向盤、排檔桿、門把、門飾板是否有污損
ABCD柱的塑料鈑件平整度是否正常，各鈑件角落的修邊是否會過於銳利有可能傷到人。
檢視車內組裝是否平順，塑膠件有無毛邊或突起，車內頂棚是否髒舊或破損
裝配是否完整，如石英鐘、除霧線、毛毯、椅套是否短缺。

車門

各個車門的開啟是否順暢無雜音，與車身鈑金件的間隙是否平均正常。

兒童安全鎖的測試
喇叭外罩與車門飾板是否安裝妥當、有無瑕疵
隔熱紙氣泡檢查

安全帶

安全帶扣上是否順暢。上下調整是否正常。
拉扯安全帶，看看接縫處是否密實。
檢查安全帶上的生產日期，不可和車籍登記的出廠日期相差超過3個月，以免買到庫存車

踏板

踩踏各板(油門、剎車、離合器)作用是否良好，是否順暢，鬆緊度與本身習慣差太多時，可請技工調整符合自己的高度。
腳踏墊是否完整無缺，另購踏墊者是否附上原廠踏墊。

中控

拉手煞車拉桿，是否正常（參考路試）
照明燈、各項行車警示燈、燈光亮度可調開關、室內燈各段測試(包含開車門、開關門的燈光延遲設計、閱讀燈、地板照明燈(有配備時)
檢查各儀表作用是否正常，儀表版燈號是否正常，再檢查開動車門窗、雨刷、中央控鎖、電動窗、車窗把手。
電動後視鏡折疊、開關是否正常，有無異音。
空調系統功能是否正常，視察冷氣作用及各段強度如何，各出風口作用是否正常。冷氣使用後，車內是否會漏水(冷凝管排水不良會漏水)車內如長期滴水，底盤鏽蝕。
音響功能是否良好正常。cd(dvd)是否挑片，收音機有無接收不良，轉台時是否會跳台。車內四(二、六)個音響喇叭是否正常，前後左右試看看。
手套箱開關測試，中央置物拖盤開合測試
點煙器功能測試，行李箱12V電源插座測試

方向盤

方向盤上下(前後)調整是否正常。
雨刷開關，每一段都測試，噴水這項也不要忘了。
若可以，測試一下除霧線，有無被貼隔熱紙廠商給切斷。

後車廂

檢查備胎是否穩固，換胎工具是否完備
備胎蓋、後檔玻璃開關是否正常

座椅

駕駛座高低調整，所有座椅前後調整，椅背角度調整，是否生鏽或產生異音。
把每一項平放或躺平方式、後座坐椅的折疊動作都測過一遍。

皮椅或布椅的平整度，是否乾淨和縫線有無脫落現象，有無外傷或瑕疵。
頭枕是否可正常調整。

前後座椅扶手。

其他

看里程數顯示

新車里程數正常約10～30公里，向外縣市調車大約50～60公里，最高不應超過80公里
若超過此數，就要詳細探查原因，若是車廠自身不斷試車，增加里程數，則顯示該車或有部分故障，須不斷試車維修

遙控器及防盜功能測試
Keyless測試
車用免持通訊系統測試

四、路試

把音響關掉
正式做路試前，請輕加油門，再踩煞車試試，確認煞車功能運作正常。
手煞車測試，可以試試拉起手煞車，並將排檔放在D檔測試有無滑動，尤其在上坡路。

慢慢拉聽有幾格聲音，應該要有7～9聲

先找個空曠安全的地方
瞭解一下愛車的煞車制動力表現。有ABS配置的車款，不用客氣，加速到時速六、七十公里，用力的踩下煞車吧…對粉多沒有使用ABS的車主來講，這會是個寶貴的使用經驗。當然，通常會由交車人員陪同測試，這時務必確認車上乘員都有綁安全帶，並告知即將做的煞車各項測試。
由最低檔往上順序排檔，注意聽聽看有沒有異常的聲音，順便瞭解一下檔位的排法。完成後可以做KICKDOWN強迫降檔測試、OD檔切換開關的變速箱反應觀察。
找路況稍有不平的地方讓車輪壓過去，仔細聽聽有無異常的聲音傳來。
找個地方做U型迴轉，試著把方向盤打到接近頂點(不能再轉動的地方再退一點點)讓車子做U型迴轉，聽看看傳動軸有沒有異常的聲音傳來。

OpenACS Configuration

2008-07-29T22:06:00.000-07:00

Install JDK
# apt-get install sun-java5-jdk
Install mysql
# apt-get install mysql-server

Install Java database (JDBC) driver
# apt-get install libmysql-java
Setting up the user to use JDBC
# export CLASSPATH=$CLASSPATH:/usr/share/java/mysql.jar
Install JBOSS (JBOSS installation guide) either way:
a) download and unzip jboss-4.2.3-GA.zip
b) download jboss-4.2.3-GA-src.tar.gz and built it using Apache ANT
c) use jems-installer:
# java -jar jems-installer-1.2.1.CR4.jar
Test the JBOSS installation
# run.sh
Make sure the followed:
a) $JBOSS_HOME, $PATH are correctly configured in bashrc file
b) /etc/hostname can be correctly resolved
$JBOSS_HOME/server/default/log would be helpful to see what's wrong with the configurations
Make sure JDBC driver mysql-connector-java.jar is located in $JBOSS_HOME/server/default/lib
Download and unpack OpenACS binary
# unzip openacs-bin-0.02.zip
Create an mysql account and database for OpenACS
# mysql -p
mysql> create database ACS;
mysql> grant all on ACS.* to openacs_user identified by ‘openacs_pw’
Deploy OpenACS
# cp openacs/openacs-ds.xml $JBOSS_HOME/server/default/deploy/
# cp openacs/acs.ear $JBOSS_HOME/server/default/deploy/
# cp openacs/openacs-service.xml $JBOSS_HOME/server/default/deploy/jms/
Modify connection-url, user-name, password in openacs-ds.xml
Run JBOSS
# run.sh
Disable JBOSS CREATE_TABLES_ON_STARTUP if getting an error "java.sql.SQLException: Table already exists"
file: $JBOSS_HOME/server/default/deploy/jms/hsqldb-jdbc2-service.xml
Web interface is accessible at http://your.server.ip:8080/openacs/index.jsf.

inotify Usage

2008-07-17T01:12:00.000-07:00

wikipedia: inotify is a Linux Kernel subsystem that provides file system event notification. It was written by John McCutchan with help from Robert Love and later Amy Griffis to replace dnotify. It was included in the mainline kernel from release 2.6.13 (2005-06-18).

Kernel configuration:
File Systems ---> [*] Inotify file change notification support

inotify parameters:
$ ls /proc/sys/fs/inotify/
max_queued_events max_user_instances max_user_watches

To install inotify-tools under Debian:
# apt-get install inotify-tools

Simple usage of inotifywait:
$ ls -F
helo helo_d/
$ inotifywait helo
Setting up watches.
Watches established.
helo OPEN
$ inotifywait -m -r helo_d
Setting up watches. Beware: since -r was given, this may take a while!
Watches established.
helo_d/ CREATE hi
helo_d/ OPEN hi
helo_d/ ATTRIB hi
helo_d/ CLOSE_WRITE,CLOSE hi
helo_d/ CREATE,ISDIR dir
helo_d/ OPEN,ISDIR dir
helo_d/ CLOSE_NOWRITE,CLOSE,ISDIR dir
helo_d/dir/ CREATE test
helo_d/dir/ OPEN test
helo_d/dir/ ATTRIB test
helo_d/dir/ CLOSE_WRITE,CLOSE test
helo_d/dir/ DELETE test

Simple usage of inotifywatch:
$ inotifywatch -r helo_d
Establishing watches...
Finished establishing watches, now collecting statistics.
total modify attrib close_write close_nowrite open moved_from moved_to create delete filename
119 4 3 5 45 50 1 1 5 5 helo_d/dir/
99 0 2 2 46 48 0 0 0 1 helo_d/

Events:

IN_ACCESS: File was accessed (read) (*)
IN_ATTRIB: Metadata changed (permissions, timestamps, extended attributes, etc.) (*)
IN_CLOSE_WRITE: File opened for writing was closed (*)
IN_CLOSE_NOWRITE: File not opened for writing was closed (*)
IN_CREATE: File/directory created in watched directory (*)
IN_DELETE: File/directory deleted from watched directory (*)
IN_DELETE_SELF: Watched file/directory was itself deleted
IN_MODIFY: File was modified (*)
IN_MOVE_SELF: Watched file/directory was itself moved
IN_MOVED_FROM: File moved out of watched directory (*)
IN_MOVED_TO: File moved into watched directory (*)
IN_OPEN: File was opened (*)

When monitoring a directory, the events marked with an asterisk (*) above can occur for files in the directory, in which case the name field in the returned inotify_event structure identifies the name of the file within the directory.

API:
int inotify_init(void);
int inotify_add_watch(int fd, const char *pathname, uint32_t mask);
int inotify_rm_watch(int fd, uint32_t wd);

Example C code:

#include 
#include 
#include 

int main(void)
{
int fd, wd, sz;
struct inotify_event buf;

if ((fd = inotify_init()) < wd =" inotify_add_watch(fd," sz =" sizeof(struct">

Reference:

Wikimedia as a presentation application

2007-05-29T17:26:00.000-07:00

I tried wikimedia slides extension at first. It worked fine. However, if you'd like to present on wikimedia, I recommend another interesting approach: one of the Firefox Greasemonkey scripts - Wikipedia Presentation.

This script automatically enables viewing any mediawiki page as a slideware presentation without modifying the page itself. Trust me, it is worth giving it a try!

Reference: mediawiki as a presentation application

Add Mantis links to ViewVC (ViewCVS)

2007-05-10T01:15:00.000-07:00

A small modification to ViewVC adding links to our mantis system:

--- lib/viewvc.py.orig  2007-05-10 10:39:10.000000000 +0800
+++ lib/viewvc.py       2007-05-10 11:17:00.000000000 +0800
@@ -973,10 +973,12 @@
 # addresses.  Note that the regexps assume the text is already HTML-encoded.
 _re_rewrite_url = re.compile('((http|https|ftp|file|svn|svn\+ssh)(://[-a-zA-Z0-9%.~:_/]+)((\?|\&amp;)([-a-zA-Z0-9%.~:_]+)=([-a-zA-Z0-9%.~:_])+)*(#([-a-zA-Z0-9%.~:_]+)?)?)')
 _re_rewrite_email = re.compile('([-a-zA-Z0-9_.\+]+)@(([-a-zA-Z0-9]+\.)+[A-Za-z]{2,4})')
+_re_rewrite_mantis = re.compile('(issue)([\ \#]+)(\d{1,7})')
 def htmlify(html):
   html = cgi.escape(html)
   html = re.sub(_re_rewrite_url, r'<a href="\1">\1</a>', html)
   html = re.sub(_re_rewrite_email, r'<a href="mailto:\1&#64;\2">\1&#64;\2</a>', html)
+  html = re.sub(_re_rewrite_mantis, r'<a href="/mantis/view.php?id=\3">\1\2\3</a>', html)
   return html

 def format_log(log, cfg):
@@ -1310,6 +1312,7 @@
   html = pp.fontify(html)
   html = re.sub(_re_rewrite_url, r'<a href="\1">\1</a>', html)
   html = re.sub(_re_rewrite_email, r'<a href="mailto:\1">\1</a>', html)
+  html = re.sub(_re_rewrite_mantis, r'<a href="/mantis/view.php?id=\3">\1\2\3</a>', html)
   return html

 def markup_stream_php(fp, cfg):

I believe it would be quite easy to understand the modification, which substitutes 'issue #id' to an URL link to mantis system.

For automatically adding a note to mantis when commit to CVS, please reference CVS Integration in Mantis manual.

Running FreeBSD over MS Windows

2007-03-28T19:28:00.000-07:00

Running FreeBSD on Microsoft Virtual PC 2007 step by step:

download and install Microsoft Virtual PC 2007
download the FreeBSD vmdk file from VMware's Virtual Appliance Marketplace - OS
download and install VMDK to VHD converter (You may need .NET Framework to run it.)

convert vmdk files to a vhd file
Create a FreeBSD virtual machine on Microsoft Virtual PC
Start FreeBSD and enjoy it!

緣起

2007-03-22T21:59:00.000-07:00

看到gen2linux在的blogger上的文章，想到好久之前也曾在這註冊了winfred，但忘了是用哪一個email註冊的… 唉，看來這個winfred.blogspot.com是要不回來了 >_<