Greetings all;
Are you send those to /dev/null? I've sent a whole bunch of them.
These crashes always seem to be preceded by a couple days of a kmail session burning up a core of 4 cores, bouncing to the next available core, at 99 to 100% at 15 second or so intervals.
These seem to be related to kmail finding trash files in its database, causing problems while re-indexing. I have two folders which are subfolders with a given years messages manually sorted into that years corpus.
When I relaunch kmail after one of these crashes, I am getting advisories that so-and-so has an index problem, and its generally the top level folder, and 2 or 3 of its subfolders that are named. And the older folders are slowly being emptied, as in messages are disappearing despite having no expiry set up.
Can anything be done, or is there something I can check-uncheck someplace?
Thanks.
Cheers, Gene Heskett
Gene Heskett wrote:
Greetings all;
Hi Gene,
Are you send those to /dev/null? I've sent a whole bunch of them.
I do not know what you are talking about, but I recall you mentioned before you had issues with your mailbox/es.
These crashes always seem to be preceded by a couple days of a kmail session burning up a core of 4 cores, bouncing to the next available core, at 99 to 100% at 15 second or so intervals.
AFAIR you are using local MailDir with tons of mail
These seem to be related to kmail finding trash files in its database, causing problems while re-indexing. I have two folders which are subfolders with a given years messages manually sorted into that years corpus.
I recall experiencing similar with KDE3 like 12y ago. Unpleasant situation for sure!
When I relaunch kmail after one of these crashes, I am getting advisories that so-and-so has an index problem, and its generally the top level folder, and 2 or 3 of its subfolders that are named. And the older folders are slowly being emptied, as in messages are disappearing despite having no expiry set up.
Can anything be done, or is there something I can check-uncheck someplace?
I don't know exactly, but I know what solved my problems was to setup imap server between the directory structure and the client. The server where the mails are is 4cores/32GB mem, however the server is not very fast - I am not sure why - I suspect the disks are not the fastest, but I do not experience any issue with mails.
Few years ago I integrated dbmail for a customer. It is extremely fast. Think about migrating those tons of mails.
I know it is not exactly an answer, but I can not think of any other. As for kmail I have not looked into it in detail, but it is complex - one has to be able to reproduce the problem to help you find a solution.
Also are you sure nothing else is using your local Maildir? I know you have many scripts for this and that.
regards
On Tuesday 15 October 2019 13:45:14 deloptes wrote:
Gene Heskett wrote:
Greetings all;
Hi Gene,
Are you send those to /dev/null? I've sent a whole bunch of them.
I do not know what you are talking about, but I recall you mentioned before you had issues with your mailbox/es.
These crashes always seem to be preceded by a couple days of a kmail session burning up a core of 4 cores, bouncing to the next available core, at 99 to 100% at 15 second or so intervals.
AFAIR you are using local MailDir with tons of mail
yes but its slowly disappearing.
These seem to be related to kmail finding trash files in its database, causing problems while re-indexing. I have two folders which are subfolders with a given years messages manually sorted into that years corpus.
I recall experiencing similar with KDE3 like 12y ago. Unpleasant situation for sure!
When I relaunch kmail after one of these crashes, I am getting advisories that so-and-so has an index problem, and its generally the top level folder, and 2 or 3 of its subfolders that are named. And the older folders are slowly being emptied, as in messages are disappearing despite having no expiry set up.
Can anything be done, or is there something I can check-uncheck someplace?
I don't know exactly, but I know what solved my problems was to setup imap server between the directory structure and the client. The server where the mails are is 4cores/32GB mem, however the server is not very fast - I am not sure why - I suspect the disks are not the fastest, but I do not experience any issue with mails.
I'm currently sucking, with fetchmail, from a dovecot database at my isp, but the procmail direct deposit into the spam folder, which gets mv'd to spam-hold for one day by the same script that has sa-learn spam being done on that folder, moving what it has looked at to spam-hold. Thats the only non-kmail accesses that I have programmed. Any other incoming mail is deposited in /var/mail, and dbus sends kmail a check local folder message for new mail by way of inotifywait seeing the closing of the mail file and triggering the dbus message. The locale /var/mail/mailfile generally has a lifetime in the non-zero length state of milliseconds.
Few years ago I integrated dbmail for a customer. It is extremely fast. Think about migrating those tons of mails.
describe that please.
I know it is not exactly an answer, but I can not think of any other. As for kmail I have not looked into it in detail, but it is complex - one has to be able to reproduce the problem to help you find a solution.
Also are you sure nothing else is using your local Maildir? I know you have many scripts for this and that.
procmail may, very occasionally place a particularly nasty bit of stuff directly in the spam folder, but thats not more than a weekly occurrence. If that often. I'll find that recipe and nuke it, just for S&G. Or better yet, send that one to /dev/null. At least I'll have a log entry. Right now, I think I'll go over the /home/gene/Mail dir and nuke every sorted index just for S&G as thats nowhere near universally present. Then I'll exit kmail as the uptime is 29+days, and restart it.
regards
Thanks
Cheers, Gene Heskett
Gene Heskett wrote:
On Tuesday 15 October 2019 13:45:14 deloptes wrote:
Gene Heskett wrote:
Greetings all;
Hi Gene,
Are you send those to /dev/null? I've sent a whole bunch of them.
I do not know what you are talking about, but I recall you mentioned before you had issues with your mailbox/es.
These crashes always seem to be preceded by a couple days of a kmail session burning up a core of 4 cores, bouncing to the next available core, at 99 to 100% at 15 second or so intervals.
AFAIR you are using local MailDir with tons of mail
yes but its slowly disappearing.
BACKUP? mails can not be that big compared to nowdays drives capacity and prices
These seem to be related to kmail finding trash files in its database, causing problems while re-indexing. I have two folders which are subfolders with a given years messages manually sorted into that years corpus.
I recall experiencing similar with KDE3 like 12y ago. Unpleasant situation for sure!
When I relaunch kmail after one of these crashes, I am getting advisories that so-and-so has an index problem, and its generally the top level folder, and 2 or 3 of its subfolders that are named. And the older folders are slowly being emptied, as in messages are disappearing despite having no expiry set up.
Can anything be done, or is there something I can check-uncheck someplace?
I don't know exactly, but I know what solved my problems was to setup imap server between the directory structure and the client. The server where the mails are is 4cores/32GB mem, however the server is not very fast - I am not sure why - I suspect the disks are not the fastest, but I do not experience any issue with mails.
I'm currently sucking, with fetchmail, from a dovecot database at my isp, but the procmail direct deposit into the spam folder, which gets mv'd to spam-hold for one day by the same script that has sa-learn spam being done on that folder, moving what it has looked at to spam-hold. Thats the only non-kmail accesses that I have programmed. Any other incoming mail is deposited in /var/mail, and dbus sends kmail a check local folder message for new mail by way of inotifywait seeing the closing of the mail file and triggering the dbus message. The locale /var/mail/mailfile generally has a lifetime in the non-zero length state of milliseconds.
This is very complicated process. Are you sure kmail is working properly. Unfortunately to inspect kmail one should compile it with Debug enabled and install the additional package. So without closer inspection it is hard to say.
I can only advise to simplify the process. This dovecot server for sure speaks imap to start with. So you could sync your own imap server and let the imap server be the interface to all your mails.
Few years ago I integrated dbmail for a customer. It is extremely fast. Think about migrating those tons of mails.
describe that please.
http://www.dbmail.org/dokuwiki/doku.php
look at the project. AFAIR it can handle postgres or mysql. I used mysql and it was really impressive. In few words it is an IMAP server with database backend. Might be suitable, but also might be too much work (see below) compared to dovecot. I found dovecot more complex, but is widely used and with a lot of documentation and support around.
I know it is not exactly an answer, but I can not think of any other. As for kmail I have not looked into it in detail, but it is complex - one has to be able to reproduce the problem to help you find a solution.
Also are you sure nothing else is using your local Maildir? I know you have many scripts for this and that.
procmail may, very occasionally place a particularly nasty bit of stuff directly in the spam folder, but thats not more than a weekly occurrence. If that often. I'll find that recipe and nuke it, just for S&G. Or better yet, send that one to /dev/null. At least I'll have a log entry. Right now, I think I'll go over the /home/gene/Mail dir and nuke every sorted index just for S&G as thats nowhere near universally present. Then I'll exit kmail as the uptime is 29+days, and restart it.
I don't know Gene. I have the suspicion that there are two things working on your maildir corrupting the files and I think you overcomplicated the things. There is a program called imapsync for example. If I were you and I wanted to save mails locally, I would use imapsync with either dovecot or dbmail and use the imap interface to read whatever mails I want with kmail - tends to work pretty well for me for over 10y now.
I use two imap servers for mail with kmail - the local one and the public one. There might be around 10000msg there at the moment - most of them on the public server. I do not sync them locally, but I was playing with this idea via imapsync. I used imapsync in one company few years ago and it is very robust application.
Probably the easiest way is to let dovecot handle your Maildir (don't forget to relocate it outside of your ~/ and let kmail forget it). I think this is what I finally ended up doing may be 10y ago. Since then 0 problems.
I hope it helps
regards
On Tuesday 15 October 2019 17:04:54 deloptes wrote:
Gene Heskett wrote:
On Tuesday 15 October 2019 13:45:14 deloptes wrote:
Gene Heskett wrote:
Greetings all;
Hi Gene,
Are you send those to /dev/null? I've sent a whole bunch of them.
I do not know what you are talking about, but I recall you mentioned before you had issues with your mailbox/es.
These crashes always seem to be preceded by a couple days of a kmail session burning up a core of 4 cores, bouncing to the next available core, at 99 to 100% at 15 second or so intervals.
AFAIR you are using local MailDir with tons of mail
yes but its slowly disappearing.
BACKUP? mails can not be that big compared to nowdays drives capacity and prices
Amanda does that every night, no complaints. I don't know the message is gone till I need to search for a senders name I know well is there as I had an extensive conversation with him in 2002. Search finds nothing, and instantly. Go look a that years folder, which should have 1000+ messages in it, and today there are only two msgs in that subfolder.
These seem to be related to kmail finding trash files in its database, causing problems while re-indexing. I have two folders which are subfolders with a given years messages manually sorted into that years corpus.
I recall experiencing similar with KDE3 like 12y ago. Unpleasant situation for sure!
When I relaunch kmail after one of these crashes, I am getting advisories that so-and-so has an index problem, and its generally the top level folder, and 2 or 3 of its subfolders that are named. And the older folders are slowly being emptied, as in messages are disappearing despite having no expiry set up.
Can anything be done, or is there something I can check-uncheck someplace?
I don't know exactly, but I know what solved my problems was to setup imap server between the directory structure and the client. The server where the mails are is 4cores/32GB mem, however the server is not very fast - I am not sure why - I suspect the disks are not the fastest, but I do not experience any issue with mails.
I'm currently sucking, with fetchmail, from a dovecot database at my isp, but the procmail direct deposit into the spam folder, which gets mv'd to spam-hold for one day by the same script that has sa-learn spam being done on that folder, moving what it has looked at to spam-hold. Thats the only non-kmail accesses that I have programmed. Any other incoming mail is deposited in /var/mail, and dbus sends kmail a check local folder message for new mail by way of inotifywait seeing the closing of the mail file and triggering the dbus message. The locale /var/mail/mailfile generally has a lifetime in the non-zero length state of milliseconds.
This is very complicated process. Are you sure kmail is working properly. Unfortunately to inspect kmail one should compile it with Debug enabled and install the additional package. So without closer inspection it is hard to say.
I can only advise to simplify the process. This dovecot server for sure speaks imap to start with. So you could sync your own imap server and let the imap server be the interface to all your mails.
Few years ago I integrated dbmail for a customer. It is extremely fast. Think about migrating those tons of mails.
describe that please.
http://www.dbmail.org/dokuwiki/doku.php
look at the project. AFAIR it can handle postgres or mysql. I used mysql and it was really impressive. In few words it is an IMAP server with database backend. Might be suitable, but also might be too much work (see below) compared to dovecot. I found dovecot more complex, but is widely used and with a lot of documentation and support around.
I know it is not exactly an answer, but I can not think of any other. As for kmail I have not looked into it in detail, but it is complex - one has to be able to reproduce the problem to help you find a solution.
Also are you sure nothing else is using your local Maildir? I know you have many scripts for this and that.
procmail may, very occasionally place a particularly nasty bit of stuff directly in the spam folder, but thats not more than a weekly occurrence. If that often. I'll find that recipe and nuke it, just for S&G. Or better yet, send that one to /dev/null. At least I'll have a log entry. Right now, I think I'll go over the /home/gene/Mail dir and nuke every sorted index just for S&G as thats nowhere near universally present. Then I'll exit kmail as the uptime is 29+days, and restart it.
I don't know Gene. I have the suspicion that there are two things working on your maildir corrupting the files and I think you overcomplicated the things. There is a program called imapsync for example. If I were you and I wanted to save mails locally, I would use imapsync with either dovecot or dbmail and use the imap interface to read whatever mails I want with kmail - tends to work pretty well for me for over 10y now.
I use two imap servers for mail with kmail - the local one and the public one. There might be around 10000msg there at the moment - most of them on the public server. I do not sync them locally, but I was playing with this idea via imapsync. I used imapsync in one company few years ago and it is very robust application.
Probably the easiest way is to let dovecot handle your Maildir (don't forget to relocate it outside of your ~/ and let kmail forget it). I think this is what I finally ended up doing may be 10y ago. Since then 0 problems.
I hope it helps
regards
To unsubscribe, e-mail: trinity-users-unsubscribe@lists.pearsoncomputing.net For additional commands, e-mail: trinity-users-help@lists.pearsoncomputing.net Read list messages on the web archive: http://trinity-users.pearsoncomputing.net/ Please remember not to top-post: http://trinity.pearsoncomputing.net/mailing_lists/#top-posting
Cheers, Gene Heskett
On Tuesday 15 October 2019 18:56:34 Gene Heskett wrote:
On Tuesday 15 October 2019 17:04:54 deloptes wrote:
Gene Heskett wrote:
On Tuesday 15 October 2019 13:45:14 deloptes wrote:
Gene Heskett wrote:
Greetings all;
Hi Gene,
Are you send those to /dev/null? I've sent a whole bunch of them.
I do not know what you are talking about, but I recall you mentioned before you had issues with your mailbox/es.
These crashes always seem to be preceded by a couple days of a kmail session burning up a core of 4 cores, bouncing to the next available core, at 99 to 100% at 15 second or so intervals.
AFAIR you are using local MailDir with tons of mail
yes but its slowly disappearing.
BACKUP? mails can not be that big compared to nowdays drives capacity and prices
Amanda does that every night, no complaints. I don't know the message is gone till I need to search for a senders name I know well is there as I had an extensive conversation with him in 2002. Search finds nothing, and instantly. Go look a that years folder, which should have 1000+ messages in it, and today there are only two msgs in that subfolder.
These seem to be related to kmail finding trash files in its database, causing problems while re-indexing. I have two folders which are subfolders with a given years messages manually sorted into that years corpus.
I recall experiencing similar with KDE3 like 12y ago. Unpleasant situation for sure!
Its doing it again since sometime yesterday. A forever session burning up a core, bouncing from core to core. I killed all the indice files a couple 3 days back, and it took several hours to rebuild, then went back to sleep for a day or so.
Lets explore the use of dbus to manage the contents of the two spam dirs. My sa-learn driver script is using system stuff to manage those dirs which kmail might not agree with, so if kmails own move or delete commands were used instead, might help it to maintain valid indices? As an experiment, I just killed the indices for those 2 dirs, and will see if that stops it in due time.
Where do I find the docs on how to do that? The help files for kmail aren't exactly voluminous on that subject.
When I relaunch kmail after one of these crashes, I am getting advisories that so-and-so has an index problem, and its generally the top level folder, and 2 or 3 of its subfolders that are named. And the older folders are slowly being emptied, as in messages are disappearing despite having no expiry set up.
Can anything be done, or is there something I can check-uncheck someplace?
I don't know exactly, but I know what solved my problems was to setup imap server between the directory structure and the client. The server where the mails are is 4cores/32GB mem, however the server is not very fast - I am not sure why - I suspect the disks are not the fastest, but I do not experience any issue with mails.
I'm currently sucking, with fetchmail, from a dovecot database at my isp, but the procmail direct deposit into the spam folder, which gets mv'd to spam-hold for one day by the same script that has sa-learn spam being done on that folder, moving what it has looked at to spam-hold. Thats the only non-kmail accesses that I have programmed. Any other incoming mail is deposited in /var/mail, and dbus sends kmail a check local folder message for new mail by way of inotifywait seeing the closing of the mail file and triggering the dbus message. The locale /var/mail/mailfile generally has a lifetime in the non-zero length state of milliseconds.
This is very complicated process. Are you sure kmail is working properly. Unfortunately to inspect kmail one should compile it with Debug enabled and install the additional package. So without closer inspection it is hard to say.
I can only advise to simplify the process. This dovecot server for sure speaks imap to start with. So you could sync your own imap server and let the imap server be the interface to all your mails.
Few years ago I integrated dbmail for a customer. It is extremely fast. Think about migrating those tons of mails.
describe that please.
http://www.dbmail.org/dokuwiki/doku.php
look at the project. AFAIR it can handle postgres or mysql. I used mysql and it was really impressive. In few words it is an IMAP server with database backend. Might be suitable, but also might be too much work (see below) compared to dovecot. I found dovecot more complex, but is widely used and with a lot of documentation and support around.
I know it is not exactly an answer, but I can not think of any other. As for kmail I have not looked into it in detail, but it is complex - one has to be able to reproduce the problem to help you find a solution.
Also are you sure nothing else is using your local Maildir? I know you have many scripts for this and that.
procmail may, very occasionally place a particularly nasty bit of stuff directly in the spam folder, but thats not more than a weekly occurrence. If that often. I'll find that recipe and nuke it, just for S&G. Or better yet, send that one to /dev/null. At least I'll have a log entry. Right now, I think I'll go over the /home/gene/Mail dir and nuke every sorted index just for S&G as thats nowhere near universally present. Then I'll exit kmail as the uptime is 29+days, and restart it.
I don't know Gene. I have the suspicion that there are two things working on your maildir corrupting the files and I think you overcomplicated the things. There is a program called imapsync for example. If I were you and I wanted to save mails locally, I would use imapsync with either dovecot or dbmail and use the imap interface to read whatever mails I want with kmail - tends to work pretty well for me for over 10y now.
I use two imap servers for mail with kmail - the local one and the public one. There might be around 10000msg there at the moment - most of them on the public server. I do not sync them locally, but I was playing with this idea via imapsync. I used imapsync in one company few years ago and it is very robust application.
Probably the easiest way is to let dovecot handle your Maildir (don't forget to relocate it outside of your ~/ and let kmail forget it). I think this is what I finally ended up doing may be 10y ago. Since then 0 problems.
I hope it helps
regards
- To unsubscribe, e-mail:
trinity-users-unsubscribe@lists.pearsoncomputing.net For additional commands, e-mail: trinity-users-help@lists.pearsoncomputing.net Read list messages on the web archive: http://trinity-users.pearsoncomputing.net/ Please remember not to top-post: http://trinity.pearsoncomputing.net/mailing_lists/#top-posting
Cheers, Gene Heskett
Cheers, Gene Heskett
Hi Gene!
Are you sure your hardware is OK?
Nik
Anno domini 2019 Tue, 15 Oct 13:00:44 -0400 Gene Heskett scripsit:
Greetings all;
Are you send those to /dev/null? I've sent a whole bunch of them.
These crashes always seem to be preceded by a couple days of a kmail session burning up a core of 4 cores, bouncing to the next available core, at 99 to 100% at 15 second or so intervals.
These seem to be related to kmail finding trash files in its database, causing problems while re-indexing. I have two folders which are subfolders with a given years messages manually sorted into that years corpus.
When I relaunch kmail after one of these crashes, I am getting advisories that so-and-so has an index problem, and its generally the top level folder, and 2 or 3 of its subfolders that are named. And the older folders are slowly being emptied, as in messages are disappearing despite having no expiry set up.
Can anything be done, or is there something I can check-uncheck someplace?
Thanks.
Cheers, Gene Heskett
On Tuesday 15 October 2019 15:01:14 Dr. Nikolaus Klepp wrote:
Hi Gene!
Are you sure your hardware is OK?
Nik
drives are fairly fresh as I always install to a new drive so I can get stuff off the old one, cmos battery replaced just a few months ago. Nothing is complaining. 29days uptime. kmail is using much of a cpu but not 100% continuous as I've no doubt it busy rebuilding indice files yet. crash at restart after sorting stuff for about a minute. crashed, reopened instantly and now I expect it will idle back when the indice recreation is finally done. the gkrellm display now looks like a normal but busy machine, no big green 100% blocks jumping from core to core.
Anno domini 2019 Tue, 15 Oct 13:00:44 -0400
Gene Heskett scripsit:
Greetings all;
Are you send those to /dev/null? I've sent a whole bunch of them.
These crashes always seem to be preceded by a couple days of a kmail session burning up a core of 4 cores, bouncing to the next available core, at 99 to 100% at 15 second or so intervals.
These seem to be related to kmail finding trash files in its database, causing problems while re-indexing. I have two folders which are subfolders with a given years messages manually sorted into that years corpus.
When I relaunch kmail after one of these crashes, I am getting advisories that so-and-so has an index problem, and its generally the top level folder, and 2 or 3 of its subfolders that are named. And the older folders are slowly being emptied, as in messages are disappearing despite having no expiry set up.
Can anything be done, or is there something I can check-uncheck someplace?
Thanks.
Cheers, Gene Heskett
Cheers, Gene Heskett
On Tuesday 15 October 2019 15:45:51 Gene Heskett wrote:
On Tuesday 15 October 2019 15:01:14 Dr. Nikolaus Klepp wrote:
Hi Gene!
Are you sure your hardware is OK?
Nik
drives are fairly fresh as I always install to a new drive so I can get stuff off the old one, cmos battery replaced just a few months ago. Nothing is complaining. 29days uptime. kmail is using much of a cpu but not 100% continuous as I've no doubt it busy rebuilding indice files yet. crash at restart after sorting stuff for about a minute. crashed, reopened instantly and now I expect it will idle back when the indice recreation is finally done. the gkrellm display now looks like a normal but busy machine, no big green 100% blocks jumping from core to core.
And after fixing dinner for us and eating it later, kmail is all done with whatever it needed to do, the cpu temp has dropped 5+F degrees, and its all happy again. I think the code at the end of each folders index regeneration needs a carefull looksee.
Anno domini 2019 Tue, 15 Oct 13:00:44 -0400
Gene Heskett scripsit:
Greetings all;
Are you send those to /dev/null? I've sent a whole bunch of them.
These crashes always seem to be preceded by a couple days of a kmail session burning up a core of 4 cores, bouncing to the next available core, at 99 to 100% at 15 second or so intervals.
These seem to be related to kmail finding trash files in its database, causing problems while re-indexing. I have two folders which are subfolders with a given years messages manually sorted into that years corpus.
When I relaunch kmail after one of these crashes, I am getting advisories that so-and-so has an index problem, and its generally the top level folder, and 2 or 3 of its subfolders that are named. And the older folders are slowly being emptied, as in messages are disappearing despite having no expiry set up.
Can anything be done, or is there something I can check-uncheck someplace?
Thanks.
Cheers, Gene Heskett
Cheers, Gene Heskett
Cheers, Gene Heskett
-
deloptes -
Dr. Nikolaus Klepp -
Gene Heskett