[REL] [CLIENT] [LOBBY] [LIVE GAME] Faster Server Browser for Titans & Classic

You can override the browser language to english now... will add other languages tomorrow.

FYI - there is a caching issue in the Uber feed where stale / historical game data is returned.

The example below shows sequential RunTime values for two test games in the Uber feed:

I'll be switching Faster Server Browser back to my filtered feed with a check for stale data soon.

This explains why you often see games initially appear, then disappear, then reappear and game settings like title alternating between old and new values.

Code:

mikeyh: 00:03:09.9770698 mikeyh2: 00:11:45.9770698
mikeyh: 00:02:45.9183368 mikeyh2: 00:11:21.9183368
mikeyh: 00:03:01.4652196 mikeyh2: 00:11:37.4652196
mikeyh: 00:03:01.4652196 mikeyh2: 00:11:37.4652196
mikeyh: 00:03:03.8558433 mikeyh2: 00:11:39.8558433
mikeyh: 00:02:58.1578720 mikeyh2: 00:11:34.1734986
mikeyh: 00:03:14.5395654 mikeyh2: 00:11:50.5395654
mikeyh: 00:03:03.8558433 mikeyh2: 00:11:39.8558433
mikeyh: 00:03:19.3364173 mikeyh2: 00:11:55.3364173
mikeyh: 00:03:08.4652238 mikeyh2: 00:11:44.4495964
mikeyh: 00:03:10.7620979 mikeyh2: 00:11:46.7620979
mikeyh: 00:03:11.2360078 mikeyh2: 00:11:47.2360078
mikeyh: 00:03:23.0864541 mikeyh2: 00:11:59.0864541
mikeyh: 00:03:13.9652241 mikeyh2: 00:11:49.9652241
mikeyh: 00:03:18.0953803 mikeyh2: 00:11:54.0953803
mikeyh: 00:03:29.5395747 mikeyh2: 00:12:05.5395747
mikeyh: 00:03:13.9652241 mikeyh2: 00:11:49.9652241
mikeyh: 00:03:18.0953803 mikeyh2: 00:11:54.0953803
mikeyh: 00:03:18.9964735 mikeyh2: 00:11:54.9964735
mikeyh: 00:03:22.1422609 mikeyh2: 00:11:58.0641376
mikeyh: 00:03:29.5395747 mikeyh2: 00:12:05.5395747
mikeyh: 00:03:35.8677058 mikeyh2: 00:12:11.8677058
mikeyh: 00:03:21.3558500 mikeyh2: 00:11:57.3558500
mikeyh: 00:03:27.5016365 mikeyh2: 00:12:03.5016365
mikeyh: 00:03:35.8677058 mikeyh2: 00:12:11.8677058
mikeyh: 00:03:29.8610165 mikeyh2: 00:12:05.8610165
mikeyh: 00:03:40.5083293 mikeyh2: 00:12:16.5083293
mikeyh: 00:03:29.8610165 mikeyh2: 00:12:05.8610165
mikeyh: 00:03:26.0433569 mikeyh2: 00:12:02.0433569
mikeyh: 00:03:33.8141443 mikeyh2: 00:12:09.8141443
mikeyh: 00:03:43.0239600 mikeyh2: 00:12:19.0239600

Faster Server Browser 2.1.0 has switched back to my filtered feed with a stale data check.

Games list is now very stable with no games appearing and disappearing.

Might also fix some of those weird join game issues...

Can someone please move over to released.

@jorgenpt might need to know that there is that kind of bug in the feed from uber.

@masterdigital knows about the issue... reported directly yesterday.

Part of the problem is that the feed is served up by multiple machines, and my suspicion is that they have a few seconds of clock skew between them. The data is requested from a shared backend, but the RunTime is calculated using local machine time. I have a proposed fix that will make this consistent.

There might also be some inconsistencies based on which machine the cloudfront request goes to, and the machine-local caches being refreshed at different times. Once RunTime is consistent I will have a better idea of where the caching is going wrong, and I can see what I can do to address it.

Thanks.

It doesn't look so bad if you ignore RunTime but I can still reproduce stale data ignoring RunTime on most test passes and it's very much a timing issue.

Are you able to insert a reliable timestamp or counter into the backend to check sequencing?

Code:

mikeyh: 00:20:04.1424238 testing8
mikeyh: 00:19:52.5211913 testing8
mikeyh: 00:20:06.2361691 testing9
mikeyh: 00:19:54.9951004 testing8
mikeyh: 00:19:54.6774472 testing9

Apologies to anyone using Faster Server Browser and https://palobby.com/ over the last 3-4 hours... looks like the feed was not updating consistently due to all the RunTime values in feed no longer being correct.

Code:

LobbyID: RunTime (first 20 in Uber feed)
13686099773578961323: -735614.22:35:09
13556627145461222022: -735614.23:19:35
8187413624850882899: -735614.23:51:01
6908691397380139796: -735615.00:13:14
17168622691451484483: -735615.00:20:44
1921998043615038756: -735615.00:24:32
11906818357856820763: -735615.00:24:36
15541660684037774455: -735615.00:25:42
3196969127337643996: -735615.00:26:37
1026302448160043147: -735615.00:28:10
5167950730862008895: -735615.00:30:32
6519728654099056787: -735615.00:31:41
2740284697065126794: -735615.00:33:17
14554547715014268923: -735615.00:34:45
14111154542225073784: -735615.00:36:01
4846758931402743913: -735615.00:37:36
13829818348123142532: -735615.00:42:05
4948753351669002670: -735615.00:42:30
3041934323236595945: -735615.00:42:43
13121709523075891702: -735615.00:43:19
4021198713374733270: -735615.00:43:53

@jorgenpt thanks for fixing the RunTime values and adding BackendTime.

Will switch back on the stale data check and run some tests.

@jorgenpt

Here's a 1 minute sample of the current feed:

Code:

FRESH 2015-01-19.23:44:37
STALE 1: 2015-01-19.23:44:28
FRESH 2015-01-19.23:44:41
FRESH 2015-01-19.23:44:43
FRESH 2015-01-19.23:44:46
FRESH 2015-01-19.23:44:50
FRESH 2015-01-19.23:44:53
STALE 1: 2015-01-19.23:44:48
STALE 2: 2015-01-19.23:44:47
FRESH 2015-01-19.23:44:55
FRESH 2015-01-19.23:45:03
STALE 1: 2015-01-19.23:44:58
STALE 2: 2015-01-19.23:44:57
FRESH 2015-01-19.23:45:06
FRESH 2015-01-19.23:45:10
STALE 1: 2015-01-19.23:45:09
FRESH 2015-01-19.23:45:11
STALE 1: 2015-01-19.23:45:08
FRESH 2015-01-19.23:45:12
FRESH 2015-01-19.23:45:20
FRESH 2015-01-19.23:45:21
FRESH 2015-01-19.23:45:24
FRESH 2015-01-19.23:45:26
FRESH 2015-01-19.23:45:30
FRESH 2015-01-19.23:45:32
STALE 1: 2015-01-19.23:45:27

@jorgenpt added some stats into my feed: https://dfpsrd4q7p23m.cloudfront.net/ajax/games/

Max consecutive stale count is currently 13+ seconds of oscillating stale data!

Code:

"BackendTime":"2015-01-20.00:22:53","gFasterServerBrowser_stale_count":11,"gFasterServerBrowser_max_stale_count":13,"gFasterServerBrowser_filtered_total":0,"gFasterServerBrowser_no_TitleData_count":0,"gFasterServerBrowser_no_system_count":0,"gFasterServerBrowser_old_game_count":0,"maxPlayers":"1235","maxGames":"630"}

When I fixed the negative runtime, the runtime should've been /less/ variable. I accidentally flipped my caching check in the code, so after I deployed a change to remove some (no longer needed) caching, it was still doing it. It was redeployed today, and now it seems to no longer be sending any stale / outdated data. Multi-level and multi-machine caches are hard problems, so less levels & sites of caching the better.

Let me know if you after this point see any stale data, and what (if any) transformations you're still doing on the feed with Faster Server Browser that we might want to include in our CurrentGames API.

Oh, and I'd be curious to hear how often you send 304 Not Modified responses instead of full responses. I would consider implementing If-Modified-Since on our API if it's significant.

I really appreciate your help in finding issues with our server browser feed.

Thanks... no stale data in the feed over a 30 minute test run.

Thanks

If you take a look at the network requests for https://palobby.com/ you'll see a reasonable amount of 304s even though BackendTime and RunTime are now constantly updating.

I'm not seeing any when running Faster Server Browser and will take a look at that next week after I finish another project.

The original intention of the expires 1s and public, must-revalidate, proxy-revalidate was to reduce constant origin queries from CloudFront within the ~1 second update interval.

I'm writing to a max gzipped file and using nginx with gzip_static to serve the games feed to CloudFront as fast as possible. Using SSL with OSCP stapling adds some overhead.

Is there any chance you can add some of the game settings like dynamic alliances back into the feed?

Is this the place to discuss PA lobby? I like that PA Chat is built into it, but there's some UI issues:

1. Need a way to get at the chat without the lobby stuff. I hear this is something. Woo!
2. Notifications aren't useful because there's too many. I want notifications when I'm spoken to but not for every message.
3. Notifications disappear and that's not useful because I'm not always in front of my PC.

palobby probably needs it's own thread soon.

1. I'm going to add a full screen, mobile optimised chat page when I have some more time later this month
2. I like all notifications, but if you don't like them turn them off with the notifications checkbox on the top right of the window (mentions always show notifications)
3. I played with different timings and 5-8 seconds allowed you to read messages in real time. You can't have too many notifications open so it can't be a chat history. Also played with not closing mentions so that they remain on screen and are dismissed manually. Mentions are now highlighted in the chat history so that might help. I managed to crash chrome on OS X with too many notifications open!

But you can't really, not without being an ******* who people speak to and never respond. There's no way to know you're being talked to without having the window permanently open or being notification spammed. Talking to people who don't talk back is tedious, so I'd rather just not be in the room than be that guy.

Reload and try now... mention notifications stay on screen.