Page 1 of 2 12 LastLast
Results 1 to 10 of 17
  1. #1
    Cake! InsaneJ's Avatar
    Join Date
    Jan 2012
    Location
    Cakeville
    Posts
    4,689
    Blog Entries
    21

    Server disconnect problem solved?

    We've, (well actually you guys) have been experiencing a lot of disconnects from the TFC server the past few days. We may have found a fix for it. Warning: techno babble ahead!


    We use Xen to run several virtual servers. It turns out that under certain workloads a bug in the xen-netfront driver is getting triggered. This causes packets to be dropped. When enough packets get dropped and the latency between a player and the server is high enough, players can get disconnected. It appears this can happen to any Linux kernel newer than 3.7. We have just upgraded the Linux kernel on our server to 3.17.7 which contains a fix for this issue.

    Please let us know if guys have a better connection with the server after this update.

  2. #2
    No fix yet it would seem...


    Still getting time outs, mostly after more than 20 players join the server.


    I would recommend lowering the number of possible players, but that would piss off a lot of people.

  3. #3
    Lazy Idle Couch Potato Heptagon_ru's Avatar
    Join Date
    Aug 2014
    Location
    Russia, near Moscow
    Posts
    1,006
    Two players blamed server broadcasts for timeouts. I just checked: 3 times the timeouts had approx 5:30 minutes between each other, twice occuring right after a broadcast text appears in chat. Probably the broadcast plugin could glitch when number of people is high, about 20?
    And yes, constant timeouts today.

  4. #4
    At peak times it's like every 5 minutes...

    I just tried to mine... I mined a few blocks -> time out -> half the blocks are reset...

    Gets pretty annoying at times.

  5. #5
    Lazy Idle Couch Potato Heptagon_ru's Avatar
    Join Date
    Aug 2014
    Location
    Russia, near Moscow
    Posts
    1,006
    It also seems that if I see the broadcast message on the screen, I'm not got timed out, just some lag for like 10 seconds, and then server continues to work, but some people are disconnected, like 3-5.
    When no message appears - a timeout.
    Just a hypothesis. Don't have enough statistical data.

  6. #6
    Cake! InsaneJ's Avatar
    Join Date
    Jan 2012
    Location
    Cakeville
    Posts
    4,689
    Blog Entries
    21
    In the server logs I see the following happening:
    Code:
    [21:39:46] [Netty IO #3/WARN]: Selector.select() returned prematurely 512 times in a row; rebuilding selector.
    [21:39:46] [Netty IO #3/INFO]: Migrated 2 channel(s) to the new Selector.
    And then everybody disconnects. It seems that a workaround for a bug in netty or java (not sure which yet) activates to prevent a crash. That is good, however the result is that the connections are dropped and everybody has to reconnect. Which isn't good obviously.

    md-5, the guy behind Spigot, sais there is nothing he can do about it: https://github.com/SpigotMC/BungeeCord/issues/455

    On netty's Github people are discussing various Linux kernels but I'm not sure if that is going to be any help to us since we're already on a newer kernel then the ones they are talking about: https://github.com/netty/netty/issues/2616

    When we know more we'll post an update.

  7. #7
    I have no idea how to run those servers...

    But does it have to be Linux? Couldn't it work with another OS?


    As I said... no idea if that is even possible...

  8. #8
    Sorry about that second announcer. It was an attempt to have a more updated announcer plugin that could replace our last announcer that we removed due to the same issue of short pauses in gameplay. I'd venture to guess that any announcer using a similar system will cause the pauses so we may just have to have plain Jane announcements that don't have fancy colors, but it's better than nothing

  9. #9
    It gets pretty much unplayable if many people are online... Mostly from 16 to 24 pm GMT+1 (my timezone). It times out every 5 minutes...

  10. #10
    Around 22:00 CET I've disabled TCP segmentation offloading and generic segmentation offloading in the virtual network driver of the VM instance, this should work around the problem, but uses more cpu power to transmit packets. Let us know if this makes any difference.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •