It could be the internet losing temporary connection - i know Virgin Media often had issues in that it would be switch paths or something and for a few seconds would cut the connection.
Most programs automatically compensate for this and have automatic retries on failed connections, meaning the user would never really notice any change - but certain programs (namely mir) would instantly kick the user because it couldn't get a response, and wouldn't bother to try again.
I think it was on Chronicles were lots of people would just get kicked, and other people would be fine - but none of them reported any down time and could instantly relog.
Not knowing which server files you're using, but if you're using Crystal then increasing the TimeOut might help.
Personally however i would never host a server on a home line. Regardless of the speed of your line, most home lines are unreliable at best.