Home/IT/Debian/4K Sectors  

Debian GNU/Linux  

 

I recently bought a new larger hard drive. I didn't do any research for this, just walked in the local comupter store and took a 1 TB silent drive. They sold me a Western Digital WD10EARS. I installed Windows 7 in the 1st 128 GB and copied my Debian Lenny one partition after another to the new drive. Then a performance nightmare become true.

Every write operation to the disk, especially with larger files such as VMware's virtual hard drives, made the computer freeze almost totally for two minutes with one minute interuption each. I started googling and realized that I was a victim of the new 4 kByte sector size of the new WD drives. The size of the physical sectors on hard drives has been 256 bytes for decades but there is a new trend to enlarge this minum allocation to 4096 bytes for more efficient data handling. This is something which was posted by the hard drive industry years ago. Windows VIsta and Windows 7 are able to handle this but obviously nobody in the GNU and Linux distribution world reacted to get fdisk up2date.

This seems to be no Linux kernel problem but of GNU's fdisk. So when I originally parted my drive I used an fdisk version which was not able to realize the 4K sector size. So every write operation caused the hard drive to re-write the same sector over and over again.

Instead of changing the hard drive and instead of starting at the beginning I decided to backup all my data and solve this issue on the fly.

My first partitioning using cfdisk resulted in this setup (Windows was installed before with its own partitioner):

# sfdisk -d /dev/sda
# partition table of /dev/sda
unit: sectors

/dev/sda1 : start=     2048, size=    204800, Id= 7, bootable
/dev/sda2 : start=   206848, size= 268228608, Id= 7
/dev/sda3 : start=268435456, size=   1992060, Id=83
/dev/sda4 : start=270427516, size=1683092549, Id= 5
/dev/sda5 : start=270427579, size=    256977, Id=83
/dev/sda6 : start=270684619, size=   8000307, Id=82
/dev/sda7 : start=278684989, size=   1991997, Id=83
/dev/sda8 : start=280677049, size=   1991997, Id=83
/dev/sda9 : start=282669109, size=  16000677, Id=83
/dev/sda10: start=298669849, size=??????????, Id=83

I can't remeber the size of the last partition, it was just "the rest of the drive".

I know that Windows has done its job right and I see that all the sectors used by the Windows installer are divisible by 2048, so this is what I need to do in Debian too. This equals 1 MB or 256 blocks a 4 kByte. Due to lack of knowledge to fdisk, it is reported as "2048 blocks 512 bytes each".

First I wrote a little shell script wich prints out the beginning and size of each partition divided by 2048:

# cat wdsec.sh
D=sda
for i in `ls /dev/${D}?* | cut -f 3 -d '/' | cut -b 4- | sort -n`
do 
START=`sfdisk -l /dev/$D -uS 2>/dev/null | tr '*' ' ' | grep "${D}${i} " | tr -s ' ' | cut -f 2 -d ' '`
SIZE=`sfdisk -l /dev/$D -uS 2>/dev/null | tr '*' ' ' | grep "${D}${i} " | tr -s ' ' | cut -f 4 -d ' '`
A=`echo "$START / 2048" | bc -l`
B=`echo "$SIZE / 64" | bc -l`
echo -e "${D}${i} \t $A \t $B"
done
# bash wdsec.sh
sda1     1.00000000000000000000   3200.00000000000000000000
sda2     101.00000000000000000000         4191072.00000000000000000000
sda3     131072.00000000000000000000      31125.93750000000000000000
sda4     132044.68554687500000000000      26298272.00000000000000000000
sda5     132044.71630859375000000000      4015.26562500000000000000
sda6     138022.02587890625000000000      250010.57812500000000000000
sda7     145834.88720703125000000000      13357033.48437500000000000000

There are some partitions missing because I deleted it meanwhile and can't rember its size anymore. But you see that the first two partitions which were made by Windows 7 are integers while the sizes made by fdisk are not.

What I did now was

  1. Shrinking the filesystem of the home partition to less than half of its size.
  2. Shrinking the size of the home partition.
  3. Calculating new sectors for the new partitions: divisible by 2048 (512 byte) blocks, 2048 blocks a 512 byte between filesystem end and next partition, 2048 blocks a 512 byte between begin of extended partition and first logical partition.
  4. Creating a new temporary partition table with sfdisk.
  5. Moving old data using tar to the new temporary partitions.
  6. Deleting the old data and partitions.
  7. Creating a second temporary partition table with sfdisk using the right geometry.
  8. Moving data using tar to the new final partitions.
  9. Deleting the temporary data and partitions.
  10. Resizing the new home partition to its maximum size.

I had more than 15 partitions during migration causing new problems so I had to copy the data to the new partitions step by step. This is not described in detail here. Also the temporary table is not shown.

Anyway, the final layout is as follows:

# sfdisk -d /dev/sda
Warning: extended partition does not start at a cylinder boundary.
DOS and Linux will interpret the contents differently.
# partition table of /dev/sda
unit: sectors

/dev/sda1 : start=     2048, size=   204800, Id= 7, bootable
/dev/sda2 : start=   206848, size=268228608, Id= 7
/dev/sda3 : start=268435456, size=  2097152, Id=83
/dev/sda4 : start=270534656, size=1682980864, Id= 5
/dev/sda5 : start=270536704, size=   262144, Id=83
/dev/sda6 : start=270800896, size=  8388608, Id=82
/dev/sda7 : start=279191552, size=  2097152, Id=83
/dev/sda8 : start=281290752, size=  2097152, Id=83
/dev/sda9 : start=283389952, size= 16777216, Id=83
/dev/sda10: start=300169216, size=1653344256, Id=83
# bash wdsec.sh
sda1     1.00000000000000000000          3200.00000000000000000000
sda2     101.00000000000000000000        4191072.00000000000000000000
sda3     131072.00000000000000000000     32768.00000000000000000000
sda4     132097.00000000000000000000     26296576.00000000000000000000
sda5     132098.00000000000000000000     4096.00000000000000000000
sda6     132227.00000000000000000000     131072.00000000000000000000
sda7     136324.00000000000000000000     32768.00000000000000000000
sda8     137349.00000000000000000000     32768.00000000000000000000
sda9     138374.00000000000000000000     262144.00000000000000000000
sda10    146567.00000000000000000000     25833504.00000000000000000000
# sfdisk -l /dev/sda

Disk /dev/sda: 121601 cylinders, 255 heads, 63 sectors/track
Warning: extended partition does not start at a cylinder boundary.
DOS and Linux will interpret the contents differently.
Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0

   Device Boot Start     End   #cyls    #blocks   Id  System
/dev/sda1   *      0+     12-     13-    102400    7  HPFS/NTFS
/dev/sda2         12+  16709-  16697- 134114304    7  HPFS/NTFS
/dev/sda3      16709+  16839-    131-   1048576   83  Linux
/dev/sda4      16840+ 121600- 104761- 841490432    5  Extended
/dev/sda5      16840+  16856-     17-    131072   83  Linux
/dev/sda6      16856+  17378-    523-   4194304   82  Linux swap / Solaris
/dev/sda7      17378+  17509-    131-   1048576   83  Linux
/dev/sda8      17509+  17640-    131-   1048576   83  Linux
/dev/sda9      17640+  18684-   1045-   8388608   83  Linux
/dev/sda10     18684+ 121600- 102916- 826672128   83  Linux
# sfdisk -l /dev/sda -uS
Disk /dev/sda: 121601 cylinders, 255 heads, 63 sectors/track
Warning: extended partition does not start at a cylinder boundary.
DOS and Linux will interpret the contents differently.
Units = sectors of 512 bytes, counting from 0

   Device Boot    Start       End   #sectors  Id  System
/dev/sda1   *      2048    206847     204800   7  HPFS/NTFS
/dev/sda2        206848 268435455  268228608   7  HPFS/NTFS
/dev/sda3     268435456 270532607    2097152  83  Linux
/dev/sda4     270534656 1953515519 1682980864   5  Extended
/dev/sda5     270536704 270798847     262144  83  Linux
/dev/sda6     270800896 279189503    8388608  82  Linux swap / Solaris
/dev/sda7     279191552 281288703    2097152  83  Linux
/dev/sda8     281290752 283387903    2097152  83  Linux
/dev/sda9     283389952 300167167   16777216  83  Linux
/dev/sda10    300169216 1953513471 1653344256  83  Linux