MSPOOL = 3,368,146 KB -- Am I doomed Mr Mainwaring?

[Ian Wade G3NRW]

I have just completed my weekly Turnpike backup, and noticed that the
MSPOOL size is now 3,368,146 KB. Back in the dim recesses of my mind I
seem to remember there is a 4GB limit on this file (and on NSPOOL, but
that one is only 1GB at the moment), and I have a horrible feeling the
world is about to end.

My mail messages go back 15 years or more, and I want to keep them.
Ideally, I would like to hive off most of them to an off-line archive,
leaving just the last 12 months or so on line, but I don't know how to
do this.

As I understand it, I can export the messages to text files,
compress/archive the old ones, delete MSPOOL, then import the latest
back into MSPOOL. But this will be a *very* long job (which maybe I
should have thought about 15 years ago).

Are there any shortcuts?

[Martin Liddle]

Post by Ian Wade G3NRW
As I understand it, I can export the messages to text files,
compress/archive the old ones, delete MSPOOL, then import the latest
back into MSPOOL. But this will be a *very* long job (which maybe I
should have thought about 15 years ago).

You are making it more complicated than necessary. After
archiving/deleting the old messages you need to rebuild the database
(possibly twice) to shrink it.

[Andy]

Post by Ian Wade G3NRW
I have just completed my weekly Turnpike backup, and noticed that the
MSPOOL size is now 3,368,146 KB. Back in the dim recesses of my mind I
seem to remember there is a 4GB limit on this file (and on NSPOOL, but
that one is only 1GB at the moment), and I have a horrible feeling the
world is about to end.

It will when you reach 4Gb

Post by Ian Wade G3NRW
My mail messages go back 15 years or more, and I want to keep them.
Ideally, I would like to hive off most of them to an off-line archive,
leaving just the last 12 months or so on line, but I don't know how to
do this.
As I understand it, I can export the messages to text files,
compress/archive the old ones, delete MSPOOL, then import the latest
back into MSPOOL. But this will be a *very* long job (which maybe I
should have thought about 15 years ago).
Are there any shortcuts?

It is most likely that you have a relatively small number of large
attachments rather than a monstrous number of short emails.

Print this out then do it...

Make a backup justincase

Follow the standard procedure for compressing/rebuilding the mailbase

If all is well, proceed...

Close news-view etc so you have the Turnpike Tree displayed. CHeck
Connect is not running.

Click once on "Turnpike" at the top of the tree.

From top menu choose Tools then Searchmessages

Check that "all email messages" is in bottom box, select it if not

Search Now

{wait}

a list appears.

if one column isn't 'size' click View Columns and add it

click the word 'size' at the top of its column to sort by size

if you prefer them the other way round click it once more

check right hand slidy bar to verify you are at the end of the list

starting with the biggest, open it and decide if you really really need
to keep the attachment. If not, export it to somewhere you'll be able to
find it in later (consider renaming it) then delete it. The
with-attachment email is moved to the wastebin and its place taken by an
attachment-free copy.

after say 25, go to the wastebin and empty it

after say 100, empty bin, close TP and rebuild the database to recover
the space.

Repeat ad nauseam.

[Jim Crowther]

In demon.ip.support.turnpike, on Sun, 11 Nov 2012 08:16:16, Ian Wade

Post by Ian Wade G3NRW
My mail messages go back 15 years or more, and I want to keep them.
Ideally, I would like to hive off most of them to an off-line archive,
leaving just the last 12 months or so on line, but I don't know how to
do this.

Make a backup of your turnpike folder, naming this folder
'2012-11-11-turnpike' or somesuch.

This will be your archive, which you can access in the future by
changing the 'Paths' in the Connect log-in screen.

Now delete all messages you wish from your current working folder and
empty the wastebasket. Re-build the Mailbase.

[Tony]

Post by Ian Wade G3NRW
As I understand it, I can export the messages to text files,
compress/archive the old ones, delete MSPOOL, then import the latest
back into MSPOOL. But this will be a *very* long job (which maybe I
should have thought about 15 years ago).
Are there any shortcuts?

There's no need to archive by export/import. Instead, just copy the
Turnpike folder and subfolders to an archive folder using Windows
Explorer, with Turnpike and Connect closed. You don't need the bin
folder (hmm, you're still using 6.03, that might not have had a bin
folder; if so, you don't need to copy the program EXE or DLL files). It
would be wise to check access to the archive at this point, as below.

I suggest you empty the wastebasket and rebuild the databases to compact
them before archiving.

Start Turnpike again and delete all messages older than 12 months from
your working mailbase. Create a search for "all my email" dated
("Received") between a date before the earliest message on the system
(say 1/1/1996) and 31 December 2011, say. NB a Turnpike bug seems to
prevent setting a date earlier than 10 years ago, so you may have to use
the Advanced IMAP search instead (e.g.
UNDELETED SINCE "1-Jan-1996" BEFORE "31-Dec-2011"). Untick "Include
deleted messages in search" (speeds up delete process). That will
collect together all the mail that you want to delete (e.g. more than 1
year old), regardless of the folders it is in. Finally, empty the
wastebasket. Do this step for each user. Then rebuild the mailbase. Your
working mailbase will now have emails received after 31 Dec 2011 only.

To access your archive, start TP Explorer and at the login dialog box
change the paths to point to the archive copy. DO NOT run Connect. When
you have finished, repeat this step but change the paths to point to the
working copy, and then you can run Connect again.

When you subsequently need to archive again, create a different archive
folder. Including the date or even just year of archiving in the folder
name would help.

Regards,

[Andy]

In message <z3tE$6c+***@hotair.localhost.invalid>, Tony
<***@hotair.demon.co.uk> wrote
[]

Post by Tony
Start Turnpike again and delete all messages older than 12 months from
your working mailbase.

[snip details]

I find that if I have a large (eg 1000) number of things in my wastebin
then telling TP to empty it doesn't work. I have to go into the
wastebin, select a few screenfuls, and manually delete them.

[Tony]

Post by Andy
I find that if I have a large (eg 1000) number of things in my wastebin
then telling TP to empty it doesn't work. I have to go into the
wastebin, select a few screenfuls, and manually delete them.

I'll test whether that problem happens here next time I archive
(January). I do find that the empty wastebasket command is incredibly
slow if you have the wastebasket folder open at the time with several
screenfulls waiting to be deleted; I make sure another folder is open
instead and right-click wastebasket/Empty in the Folders pane. This is
with TP running in XP Mode under Win7/64-bit.

[J. P. Gilliver (John)]

Post by Andy
[]

Post by Tony
Start Turnpike again and delete all messages older than 12 months from
your working mailbase.

[snip details]
I find that if I have a large (eg 1000) number of things in my wastebin
then telling TP to empty it doesn't work. I have to go into the
wastebin, select a few screenfuls, and manually delete them.

I also find there are two sorts of things that can be in the WB - one
that (I think) _is_ deleted when I quit Turnpike, another that isn't:

Some while ago, I asked here if there was any way I could stop TP from
keeping a copy of everything I email or post. (If I _want_ to keep a
copy, I'd Cc: myself, for emails anyway.) Someone suggested (the general
consensus was that there's no way to stop TP making copies as such)
changing the settings so that TP stores the copies in the WB rather than
the default "Filed". This worked inasmuch as the copies do indeed go
into the WB, but they're _not_ deleted when I close TP; if I look into
the WB, there is a distinction - one sort (I think it's the ones that
_are_ going to be deleted) are shown in grey, the other in black.

From time to time I have a WB-emptying session.

[Ian]

Post by J. P. Gilliver (John)

Post by Andy
[]

Post by Tony
Start Turnpike again and delete all messages older than 12 months from
your working mailbase.

[snip details]
I find that if I have a large (eg 1000) number of things in my
wastebin then telling TP to empty it doesn't work. I have to go into
the wastebin, select a few screenfuls, and manually delete them.

I also find there are two sorts of things that can be in the WB - one
Some while ago, I asked here if there was any way I could stop TP from
keeping a copy of everything I email or post. (If I _want_ to keep a
copy, I'd Cc: myself, for emails anyway.) Someone suggested (the
general consensus was that there's no way to stop TP making copies as
such) changing the settings so that TP stores the copies in the WB
rather than the default "Filed". This worked inasmuch as the copies do
indeed go into the WB, but they're _not_ deleted when I close TP; if I
look into the WB, there is a distinction - one sort (I think it's the
ones that _are_ going to be deleted) are shown in grey, the other in black.
From time to time I have a WB-emptying session.

In Folder Properties/Filing, there is an option to turn off filing for
that folder, or to choose where to file.

Is that not what you want?

[J. P. Gilliver (John)]

[]

Post by Ian

Post by J. P. Gilliver (John)
Some while ago, I asked here if there was any way I could stop TP from
keeping a copy of everything I email or post. (If I _want_ to keep a
copy, I'd Cc: myself, for emails anyway.) Someone suggested (the
general consensus was that there's no way to stop TP making copies as
such) changing the settings so that TP stores the copies in the WB
rather than the default "Filed". This worked inasmuch as the copies do
indeed go into the WB, but they're _not_ deleted when I close TP; if I
look into the WB, there is a distinction - one sort (I think it's the
ones that _are_ going to be deleted) are shown in grey, the other in black.
From time to time I have a WB-emptying session.

In Folder Properties/Filing, there is an option to turn off filing for
that folder, or to choose where to file.
Is that not what you want?

Could be! I've just set it to not file, and we'll see if this post gets
copied.

[J. P. Gilliver (John)]

Post by J. P. Gilliver (John)
[]

Post by Ian

Post by J. P. Gilliver (John)
Some while ago, I asked here if there was any way I could stop TP
from keeping a copy of everything I email or post. (If I _want_ to
keep a copy, I'd Cc: myself, for emails anyway.) Someone suggested
(the general consensus was that there's no way to stop TP making
copies as such) changing the settings so that TP stores the copies in
the WB rather than the default "Filed". This worked inasmuch as the
copies do indeed go into the WB, but they're _not_ deleted when I
close TP; if I look into the WB, there is a distinction - one sort (I
think it's the ones that _are_ going to be deleted) are shown in grey, the other in black.
From time to time I have a WB-emptying session.

In Folder Properties/Filing, there is an option to turn off filing for
that folder, or to choose where to file.
Is that not what you want?

Could be! I've just set it to not file, and we'll see if this post gets
copied.

Nope - copy in Wastebasket as usual. (And I did make sure the post had
actually been posted.)

[Mike Barnes]

Post by Andy
I find that if I have a large (eg 1000) number of things in my wastebin
then telling TP to empty it doesn't work. I have to go into the
wastebin, select a few screenfuls, and manually delete them.

I wonder why that is.

You've just reminded me that it's a long time since I cleared old stuff
from the wastebasket, so I deleted everything over a couple of months
old. Just shy of 2000 items, deleted in one go, taking less than five
seconds.

[Paul Hyett]

Post by Mike Barnes

Post by Andy
I find that if I have a large (eg 1000) number of things in my wastebin
then telling TP to empty it doesn't work. I have to go into the
wastebin, select a few screenfuls, and manually delete them.

I wonder why that is.
You've just reminded me that it's a long time since I cleared old stuff
from the wastebasket, so I deleted everything over a couple of months
old. Just shy of 2000 items, deleted in one go, taking less than five
seconds.

Shouldn't the wastebasket automatically clear when you exit the program?

[Ian Jackson]

Post by Paul Hyett

Post by Mike Barnes

Post by Andy
I find that if I have a large (eg 1000) number of things in my wastebin
then telling TP to empty it doesn't work. I have to go into the
wastebin, select a few screenfuls, and manually delete them.

I wonder why that is.
You've just reminded me that it's a long time since I cleared old stuff
from the wastebasket, so I deleted everything over a couple of months
old. Just shy of 2000 items, deleted in one go, taking less than five
seconds.

Shouldn't the wastebasket automatically clear when you exit the program?

Only if you choose that option.

[Mike Barnes]

Post by Paul Hyett

Post by Mike Barnes

Post by Andy
I find that if I have a large (eg 1000) number of things in my wastebin
then telling TP to empty it doesn't work. I have to go into the
wastebin, select a few screenfuls, and manually delete them.

I wonder why that is.
You've just reminded me that it's a long time since I cleared old stuff
from the wastebasket, so I deleted everything over a couple of months
old. Just shy of 2000 items, deleted in one go, taking less than five
seconds.

Shouldn't the wastebasket automatically clear when you exit the program?

It's optional. I prefer not.

[Andy]

Post by Mike Barnes

Post by Paul Hyett

Post by Mike Barnes

Post by Andy
I find that if I have a large (eg 1000) number of things in my wastebin
then telling TP to empty it doesn't work. I have to go into the
wastebin, select a few screenfuls, and manually delete them.

I wonder why that is.
You've just reminded me that it's a long time since I cleared old stuff
from the wastebasket, so I deleted everything over a couple of months
old. Just shy of 2000 items, deleted in one go, taking less than five
seconds.

Shouldn't the wastebasket automatically clear when you exit the program?

It's optional. I prefer not.

There was some reason - maybe back in TP5 days - for not selecting
auto-empty. Maybe it made some problem-diagnosis difficult?

The more common reason is protection against deleting your master-work!

[Ian Wade G3NRW]

___Original Message_________________________________________

From: Ian Wade G3NRW <g3nrw-***@ntlworld.com>
Date: Sun, 11 Nov 2012 Time: 08:16:16

I have a horrible feeling the world is about to end.

Thanks everyone for the very helpful messages. I hadn't realised that
you could reduce the size of the spool files by deleting messages then
rebuilding the databases twice. I had thought that the files just grew
and grew, and the only way to reduce their size was to delete them and
start again.

That gives me hope. I estimate I have about a month to go before
Armageddon.

[J. P. Gilliver (John)]

Post by Ian Wade G3NRW
___Original Message_________________________________________
Date: Sun, 11 Nov 2012 Time: 08:16:16

I have a horrible feeling the world is about to end.

Thanks everyone for the very helpful messages. I hadn't realised that
you could reduce the size of the spool files by deleting messages then
rebuilding the databases twice. I had thought that the files just grew
and grew, and the only way to reduce their size was to delete them and
start again.
That gives me hope. I estimate I have about a month to go before
Armageddon.

I'd add my encouragement to sorting by size and then exporting
attachments; IMO keeping (any, but certainly large) attachments _inside_
emails not only slows down TP (as well as making the 'base files big),
but is also slightly more risky for the attachments: you're relying on
TP _as well as_ the OS to get at them, whereas if you export them,
you're only relying on the OS. (This is assuming you _do_ consider the
attachment important; if not, delete it altogether!) You _don't_ have to
delete the email: you can export the attachment, then remove it from the
email. (TP will replace it in the email by a text line saying where you
exported it.)

[Roy Brown]

In message <***@ntlworld.com>, Ian Wade G3NRW
<g3nrw-***@ntlworld.com> writing at 11:46:14 in his/her local time
opines:-

Post by Ian Wade G3NRW
___Original Message_________________________________________
Date: Sun, 11 Nov 2012 Time: 08:16:16

I have a horrible feeling the world is about to end.

Thanks everyone for the very helpful messages. I hadn't realised that you
could reduce the size of the spool files by deleting messages then
rebuilding the databases twice. I had thought that the files just grew and
grew, and the only way to reduce their size was to delete them and start
again.
That gives me hope. I estimate I have about a month to go before
Armageddon.

?

I estimate 3 years and 8 months, based on the numbers you quote.

Years Current Size Growth per Year
15 3,368,146 224,543
Max Size (Max - Curr)/Growth per Year
4,194,303 3.679

Or is this too simplistic?

Also, you don't say if you do regularly rebuild your TP databases or
not.
If you simply do a rebuild, what does MSPOOL come down to?

[J. P. Gilliver (John)]

Post by Roy Brown
opines:-

[]

Post by Roy Brown

Post by Ian Wade G3NRW
That gives me hope. I estimate I have about a month to go before
Armageddon.

?
I estimate 3 years and 8 months, based on the numbers you quote.
Years Current Size Growth per Year
15 3,368,146 224,543
Max Size (Max - Curr)/Growth per Year
4,194,303 3.679
Or is this too simplistic?

It doesn't take account of the increase in the average size of
attachments (I'd guess 15 years ago most were on dialup), and the
reduction in snippage (linked to the increase in top-posting - most
top-posters don't snip, which is its real disadvantage, rather than the
top-posting itself).

Post by Roy Brown
Also, you don't say if you do regularly rebuild your TP databases or
not.
If you simply do a rebuild, what does MSPOOL come down to?

(I was wondering that too. Might be worth doing twice too, especially if
the first time reports any problems.)

[Ian Wade G3NRW]

A further sign that Armageddon is nigh.

Until today, I had 102 mailing list folders set up (sad, I know), and
everything worked fine. This morning I set up the 103rd. The mailing
list folder appeared to be created as normal, but subsequent incoming
emails were not routed to that folder.

Solution: I deleted about 20 of the most unused folders. Routing to the
new folder seems to be working OK now.

Bottom line: it looks like 102 mailing list folders is the limit.

[Tony]

Post by Ian Wade G3NRW
A further sign that Armageddon is nigh.
Until today, I had 102 mailing list folders set up (sad, I know), and
everything worked fine. This morning I set up the 103rd. The mailing
list folder appeared to be created as normal, but subsequent incoming
emails were not routed to that folder.
Solution: I deleted about 20 of the most unused folders. Routing to the
new folder seems to be working OK now.
Bottom line: it looks like 102 mailing list folders is the limit.

More likely a limit in v6.03 for the space available for mailing list
rules. It was increased significantly for v6.07.

[Ian Wade G3NRW]

___Original Message_________________________________________
From: Tony <***@hotair.demon.co.uk>
Date: Sat, 17 Nov 2012 Time: 13:40:19

Post by Tony
More likely a limit in v6.03 for the space available for mailing list
rules. It was increased significantly for v6.07.

Makes sense. But I'm quite happy with 6.03.

Having 100+ mailing lists was a bit OTT anyway, as I only skim very
briefly through most of them each day.

[J. P. Gilliver (John)]

Post by Ian Wade G3NRW
A further sign that Armageddon is nigh.
Until today, I had 102 mailing list folders set up (sad, I know), and
everything worked fine. This morning I set up the 103rd. The mailing
list folder appeared to be created as normal, but subsequent incoming
emails were not routed to that folder.
Solution: I deleted about 20 of the most unused folders. Routing to the

Did you also send 20 unsubscribe emails (or whatever is the relevant
procedure in each case)? [Or does Turnpike do that when you delete such
a folder? I suspect not.] Of course [unless TP does that] I suppose
you'll still get those emails in your ordinary inbox or whatever, but if
they're little-used lists, you won't necessarily have a list of them all
...

Post by Ian Wade G3NRW
new folder seems to be working OK now.
Bottom line: it looks like 102 mailing list folders is the limit.

[Ian Wade G3NRW]

___Original Message_________________________________________
From: "J. P. Gilliver (John)" <***@nospam.demon.co.uk>
Date: Sat, 17 Nov 2012 Time: 13:57:02

Post by J. P. Gilliver (John)
Did you also send 20 unsubscribe emails (or whatever is the relevant
procedure in each case)?

No, I didn't need to, because I had already unsubscribed. The folders
were just sitting there with old messages in them. I should have cleaned
up a long time ago, but it's so hard to let go!