missing free space

On a partition mounted as /tmp on /dev/hda2 df report

df -h | grep -E "Size|hda2"
Filesystem Size Used Avail Use% Mounted on
/dev/hda2 479M 454M 0 100% /tmp

and total space used by /tmp directory is

du -sh /tmp/
1.4M /tmp/

Why this differences between used space reported df and du appear ?

--

0

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

missing free space

On Sun, Dec 31, 2006 at 01:13:30AM +0200, ccostin wrote:
> On a partition mounted as /tmp on /dev/hda2 df report
>
> df -h | grep -E "Size|hda2"
> Filesystem Size Used Avail Use% Mounted on
> /dev/hda2 479M 454M 0 100% /tmp
>
> and total space used by /tmp directory is
>
> du -sh /tmp/
> 1.4M /tmp/
>
> Why this differences between used space reported df and du appear ?
>
The way that du measure disk utilization is to check for all existing
files and directories and see how many disk blocks they use and add them
all up. OTOH, df checks with the filesystem to see how many blocks are
not used. You are asking why the sum of blocks used by existing files
and directories and the unused blocks is not the whole of the blocks in
the filesystem. The reason is that there are nonexistent files.

What that means is that a program needs a temporary file and so it calls
a function like tmpfile, which creates a temporary file. In order to
minimize the vulnerability of information leaking from a temporary file,
some programs call unlink() right after the file is created. As long as
the program retains the open file descriptor, the file can be read from
and written to, but no one else can access it, since it is not
addressable via the filesystem. Once the program closes, the file
descriptor is reclaimed by the OS and all the blocks are again marked as
free.

I believe that qemu does this as I often see the exact same thing you
are talking about after laeving qemu running for long periods of time.

Regards,

-Roberto
--
Roberto C. Sanchez
http://people.connexer.com/~roberto
http://www.connexer.com

missing free space

On 12/31/06, Roberto C. Sanchez wrote:
> On Sun, Dec 31, 2006 at 01:13:30AM +0200, ccostin wrote:
> > On a partition mounted as /tmp on /dev/hda2 df report
> >
> > df -h | grep -E "Size|hda2"
> > Filesystem Size Used Avail Use% Mounted on
> > /dev/hda2 479M 454M 0 100% /tmp
> >
> > and total space used by /tmp directory is
> >
> > du -sh /tmp/
> > 1.4M /tmp/
> >
> > Why this differences between used space reported df and du appear ?
> >
> The way that du measure disk utilization is to check for all existing
> files and directories and see how many disk blocks they use and add them
> all up. OTOH, df checks with the filesystem to see how many blocks are
> not used. You are asking why the sum of blocks used by existing files
> and directories and the unused blocks is not the whole of the blocks in
> the filesystem. The reason is that there are nonexistent files.
>
> What that means is that a program needs a temporary file and so it calls
> a function like tmpfile, which creates a temporary file. In order to
> minimize the vulnerability of information leaking from a temporary file,
> some programs call unlink() right after the file is created. As long as
> the program retains the open file descriptor, the file can be read from
> and written to, but no one else can access it, since it is not
> addressable via the filesystem. Once the program closes, the file
> descriptor is reclaimed by the OS and all the blocks are again marked as
> free.
>
> I believe that qemu does this as I often see the exact same thing you
> are talking about after laeving qemu running for long periods of time.
>

For this moment I don't run any qemu instace. But is somenthing similar.
When I test an example from php5-imagick package
(/usr/share/doc/php5-imagick/examples/resize.php)
apache2 "eat" a lot of disk space (a bug ?) from tmp
Output from lsof

COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME
apache2 10673 apache 11u REG 3,2 280080000
0 6161 /tmp/magick-XXuayBEU (deleted)

ls /tmp/magick-XXuayBEU
ls: /tmp/magick-XXuayBEU: No such file or directory

After restarting apache2 all things get back to normal, occupied
space is released.

df -h | grep -E "Size|tmp"
Filesystem Size Used Avail Use% Mounted on
/dev/hda2 479M 9.9M 444M 3% /tmp

--

missing free space

On Sun, Dec 31, 2006 at 03:09:34AM +0200, ccostin wrote:
>
> For this moment I don't run any qemu instace. But is somenthing similar.
> When I test an example from php5-imagick package
> (/usr/share/doc/php5-imagick/examples/resize.php)
> apache2 "eat" a lot of disk space (a bug ?) from tmp
> Output from lsof
>
> COMMAND PID USER FD TYPE DEVICE SIZE NODE
> NAME
> apache2 10673 apache 11u REG 3,2 280080000
> 0 6161 /tmp/magick-XXuayBEU (deleted)
>
> ls /tmp/magick-XXuayBEU
> ls: /tmp/magick-XXuayBEU: No such file or directory
>
>
> After restarting apache2 all things get back to normal, occupied
> space is released.
>
No. This is not a bug. This is exactly the behavior I was talking
about. The temporary file is created and then immediately deleted. So,
it still takes up blocks on the file system, but du has no way of
counting those blocks since the file is "gone", but it is in fact still
there since an open file descriptor exists to it.

Regards,

-Roberto

--
Roberto C. Sanchez
http://people.connexer.com/~roberto
http://www.connexer.com

missing free space

Roberto C. Sanchez wrote:
> On Sun, Dec 31, 2006 at 03:09:34AM +0200, ccostin wrote:
>> For this moment I don't run any qemu instace. But is somenthing similar.
>> When I test an example from php5-imagick package
>> (/usr/share/doc/php5-imagick/examples/resize.php)
>> apache2 "eat" a lot of disk space (a bug ?) from tmp
>> Output from lsof
>>
>> COMMAND PID USER FD TYPE DEVICE SIZE NODE
>> NAME
>> apache2 10673 apache 11u REG 3,2 280080000
>> 0 6161 /tmp/magick-XXuayBEU (deleted)
>>
>> ls /tmp/magick-XXuayBEU
>> ls: /tmp/magick-XXuayBEU: No such file or directory
>>
>>
>> After restarting apache2 all things get back to normal, occupied
>> space is released.
>>
> No. This is not a bug. This is exactly the behavior I was talking
> about. The temporary file is created and then immediately deleted. So,
> it still takes up blocks on the file system, but du has no way of
> counting those blocks since the file is "gone", but it is in fact still
> there since an open file descriptor exists to it.
>

Good explanation.

--

Syndicate content