Debian GNU/Linux
4K Sectors
I recently bought a new larger hard drive. I didn't do any research for this, just walked in the local comupter store and took a 1 TB silent drive. They sold me a Western Digital WD10EARS. I installed Windows 7 in the 1st 128 GB and copied my Debian Lenny one partition after another to the new drive. Then a performance nightmare become true.
Every write operation to the disk, especially with larger files such as VMware's virtual hard drives, made the computer freeze almost totally for two minutes with one minute interuption each. I started googling and realized that I was a victim of the new 4 kByte sector size of the new WD drives. The size of the physical sectors on hard drives has been 256 bytes for decades but there is a new trend to enlarge this minum allocation to 4096 bytes for more efficient data handling. This is something which was posted by the hard drive industry years ago. Windows VIsta and Windows 7 are able to handle this but obviously nobody in the GNU and Linux distribution world reacted to get fdisk up2date.
This seems to be no Linux kernel problem but of GNU's fdisk. So when I originally parted my drive I used an fdisk version which was not able to realize the 4K sector size. So every write operation caused the hard drive to re-write the same sector over and over again.
Instead of changing the hard drive and instead of starting at the beginning I decided to backup all my data and solve this issue on the fly.
My first partitioning using cfdisk resulted in this setup (Windows was installed before with its own partitioner):
# sfdisk -d /dev/sda # partition table of /dev/sda unit: sectors /dev/sda1 : start= 2048, size= 204800, Id= 7, bootable /dev/sda2 : start= 206848, size= 268228608, Id= 7 /dev/sda3 : start=268435456, size= 1992060, Id=83 /dev/sda4 : start=270427516, size=1683092549, Id= 5 /dev/sda5 : start=270427579, size= 256977, Id=83 /dev/sda6 : start=270684619, size= 8000307, Id=82 /dev/sda7 : start=278684989, size= 1991997, Id=83 /dev/sda8 : start=280677049, size= 1991997, Id=83 /dev/sda9 : start=282669109, size= 16000677, Id=83 /dev/sda10: start=298669849, size=??????????, Id=83
I can't remeber the size of the last partition, it was just "the rest of the drive".
I know that Windows has done its job right and I see that all the sectors used by the Windows installer are divisible by 2048, so this is what I need to do in Debian too. This equals 1 MB or 256 blocks a 4 kByte. Due to lack of knowledge to fdisk, it is reported as "2048 blocks 512 bytes each".
First I wrote a little shell script wich prints out the beginning and size of each partition divided by 2048:
# cat wdsec.sh
D=sda
for i in `ls /dev/${D}?* | cut -f 3 -d '/' | cut -b 4- | sort -n`
do
START=`sfdisk -l /dev/$D -uS 2>/dev/null | tr '*' ' ' | grep "${D}${i} " | tr -s ' ' | cut -f 2 -d ' '`
SIZE=`sfdisk -l /dev/$D -uS 2>/dev/null | tr '*' ' ' | grep "${D}${i} " | tr -s ' ' | cut -f 4 -d ' '`
A=`echo "$START / 2048" | bc -l`
B=`echo "$SIZE / 64" | bc -l`
echo -e "${D}${i} \t $A \t $B"
done
# bash wdsec.sh sda1 1.00000000000000000000 3200.00000000000000000000 sda2 101.00000000000000000000 4191072.00000000000000000000 sda3 131072.00000000000000000000 31125.93750000000000000000 sda4 132044.68554687500000000000 26298272.00000000000000000000 sda5 132044.71630859375000000000 4015.26562500000000000000 sda6 138022.02587890625000000000 250010.57812500000000000000 sda7 145834.88720703125000000000 13357033.48437500000000000000
There are some partitions missing because I deleted it meanwhile and can't rember its size anymore. But you see that the first two partitions which were made by Windows 7 are integers while the sizes made by fdisk are not.
What I did now was
- Shrinking the filesystem of the home partition to less than half of its size.
- Shrinking the size of the home partition.
- Calculating new sectors for the new partitions: divisible by 2048 (512 byte) blocks, 2048 blocks a 512 byte between filesystem end and next partition, 2048 blocks a 512 byte between begin of extended partition and first logical partition.
- Creating a new temporary partition table with sfdisk.
- Moving old data using tar to the new temporary partitions.
- Deleting the old data and partitions.
- Creating a second temporary partition table with sfdisk using the right geometry.
- Moving data using tar to the new final partitions.
- Deleting the temporary data and partitions.
- Resizing the new home partition to its maximum size.
I had more than 15 partitions during migration causing new problems so I had to copy the data to the new partitions step by step. This is not described in detail here. Also the temporary table is not shown.
Anyway, the final layout is as follows:
# sfdisk -d /dev/sda Warning: extended partition does not start at a cylinder boundary. DOS and Linux will interpret the contents differently. # partition table of /dev/sda unit: sectors /dev/sda1 : start= 2048, size= 204800, Id= 7, bootable /dev/sda2 : start= 206848, size=268228608, Id= 7 /dev/sda3 : start=268435456, size= 2097152, Id=83 /dev/sda4 : start=270534656, size=1682980864, Id= 5 /dev/sda5 : start=270536704, size= 262144, Id=83 /dev/sda6 : start=270800896, size= 8388608, Id=82 /dev/sda7 : start=279191552, size= 2097152, Id=83 /dev/sda8 : start=281290752, size= 2097152, Id=83 /dev/sda9 : start=283389952, size= 16777216, Id=83 /dev/sda10: start=300169216, size=1653344256, Id=83
# bash wdsec.sh sda1 1.00000000000000000000 3200.00000000000000000000 sda2 101.00000000000000000000 4191072.00000000000000000000 sda3 131072.00000000000000000000 32768.00000000000000000000 sda4 132097.00000000000000000000 26296576.00000000000000000000 sda5 132098.00000000000000000000 4096.00000000000000000000 sda6 132227.00000000000000000000 131072.00000000000000000000 sda7 136324.00000000000000000000 32768.00000000000000000000 sda8 137349.00000000000000000000 32768.00000000000000000000 sda9 138374.00000000000000000000 262144.00000000000000000000 sda10 146567.00000000000000000000 25833504.00000000000000000000
# sfdisk -l /dev/sda Disk /dev/sda: 121601 cylinders, 255 heads, 63 sectors/track Warning: extended partition does not start at a cylinder boundary. DOS and Linux will interpret the contents differently. Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0 Device Boot Start End #cyls #blocks Id System /dev/sda1 * 0+ 12- 13- 102400 7 HPFS/NTFS /dev/sda2 12+ 16709- 16697- 134114304 7 HPFS/NTFS /dev/sda3 16709+ 16839- 131- 1048576 83 Linux /dev/sda4 16840+ 121600- 104761- 841490432 5 Extended /dev/sda5 16840+ 16856- 17- 131072 83 Linux /dev/sda6 16856+ 17378- 523- 4194304 82 Linux swap / Solaris /dev/sda7 17378+ 17509- 131- 1048576 83 Linux /dev/sda8 17509+ 17640- 131- 1048576 83 Linux /dev/sda9 17640+ 18684- 1045- 8388608 83 Linux /dev/sda10 18684+ 121600- 102916- 826672128 83 Linux
# sfdisk -l /dev/sda -uS Disk /dev/sda: 121601 cylinders, 255 heads, 63 sectors/track Warning: extended partition does not start at a cylinder boundary. DOS and Linux will interpret the contents differently. Units = sectors of 512 bytes, counting from 0 Device Boot Start End #sectors Id System /dev/sda1 * 2048 206847 204800 7 HPFS/NTFS /dev/sda2 206848 268435455 268228608 7 HPFS/NTFS /dev/sda3 268435456 270532607 2097152 83 Linux /dev/sda4 270534656 1953515519 1682980864 5 Extended /dev/sda5 270536704 270798847 262144 83 Linux /dev/sda6 270800896 279189503 8388608 82 Linux swap / Solaris /dev/sda7 279191552 281288703 2097152 83 Linux /dev/sda8 281290752 283387903 2097152 83 Linux /dev/sda9 283389952 300167167 16777216 83 Linux /dev/sda10 300169216 1953513471 1653344256 83 Linux