I'm starting to HATE this game...

[_____________fearGod]

I've been playing since beta and cannot believe I'm still dying in missions, due to being locked inside!
I've got semi-high leveled 5 professions and recently started a Martial Artist: I'm fighting hard and trying to find something worthwhile, in the "newly" tweaked chests...I need 3-4k to level and considered leaving the mission to save, as most mobs are deep orange - red.
Figuring that the crawl is fun, I press on...

BAMMM !!!

I run into a mob (red) and he's actually "warping" around in the room. Eventually he agros another mob and I've got 2 on me.
Playing it safe, I make for the exit (95% of my experience is unprotected).
When I get into the main room, every exit has a "mirrored" door.
I tried every door for 30 seconds, before /quitting out of the game.

I haven't logged in...I know I'm dead... I know I've wasted my night, but I had to come here and say; I really enjoyed this game, but recently the warping mobs, lag, and now this ancient issue is pissing me off more and more.
I cannot wait for another MMORPG so I can bail.

Additionally, missions aren't quests and an RPG without quests ISN'T an RPG.
"Questing with an AO twist" is taking FAR too long.

peace out

[Sorzella]

Good I am glad Im not the only one having this problem.. As you say above, I also have had this problem since release of the game.. Ive become all to familiar with it and have actually tested it and reproduced it at will..

It seems that it happens 95% of the time when a large mission area is mostly or partially cleared out. Say you are very far from the exit, youve worked your way there and decide to save. If you run for the exit through many many rooms, its like the server does not comprehend it, like your partially LD or something.. On the client, your running through the doors as far as you can till you reach the end of the grid or whatever(im not a programmer).. All of a sudden most of the doors are blurred because your client has just realized that it is not in sync with the server. Meanwhile on the server your still back some rooms.. 50% of the time, if you actually run back the way you came to the spot that your originally started on this trek, the client and server will resync and you can then go for the exit again.. Ive tested this every time it happens to me and half the time it works.. Its too bad though that the other half of the time, whatever I am running from has got me..

This has happend to me both on Cable and Dial up many many times...

This has happend to me on 4 different PCs with all different hardware in each one..

This Bug has existed since launch..


Another thing I would like to add here in case anyone is reading this, is when in grouped missions, people tend to become invisible to each other.. One minute 1 group member cant see 3, next minute 4 group members cant see 1 etc, etc..... This is getting tiresome as well..


I wonder if Funcom even has any intention of looking into the matter or if they know, are they going to fix it anytime here soon?

Sorz

[Myga]

They need to start using UDP packets like every other game instead of TCP to send the data IMO. At least make it an option. Having programmed a bit of network code, I am 90% sure that TCP is the reason its so hard for them to fix the sync problems.

[___________Damion666]

Originally posted by fearGod

I cannot wait for another MMORPG so I can bail.

Hehe.... i whole heartly agree. I just dont wanna hit the button until i have something else (everything else out atm just plain sucks (even a good fps (sof 2) would do it ;p)).

[___________Zeroshift]

I quit due to technical reasons too. Boring gameplay was a contributer, but at least I was willing to hold on until the next few patches, but what's the point if I get disconnected every 5 minutes.. I got less than a week on my subscription and I seriously doubt FC will fix in a week what they couldn't fix in 6 months.

[________________Seid]

Originally posted by MSag
They need to start using UDP packets like every other game instead of TCP to send the data IMO. At least make it an option. Having programmed a bit of network code, I am 90% sure that TCP is the reason its so hard for them to fix the sync problems.

While a programmer myself I have not programmed much low level network code...

What is the advantage of UDP over TCP ?

TCP has flow control while UDP does not right ?

I recall once upon a time someone made a file transfer program based on UDP, called FSP, which was more or less banned a while due to no flow control and bad traffic management...it basically kept pumping packets out until it got a response...

A so-called "connection-less" file transfer.

This was around 7 years ago tho.

Messaging programs tend to use UDP for messages and TCP for filetransfer I think...

How does using UDP help fixing sync problems when it does not have flow control ?

By accepting data loss and not bother to retransmit ?

[Myga]

Originally posted by Seid


While a programmer myself I have not programmed much low level network code...

What is the advantage of UDP over TCP ?

TCP has flow control while UDP does not right ?

I recall once upon a time someone made a file transfer program based on UDP, called FSP, which was more or less banned a while due to no flow control and bad traffic management...it basically kept pumping packets out until it got a response...

A so-called "connection-less" file transfer.

This was around 7 years ago tho.

Messaging programs tend to use UDP for messages and TCP for filetransfer I think...

How does using UDP help fixing sync problems when it does not have flow control ?

By accepting data loss and not bother to retransmit ?

More or less yes, problem is TCP guarantees data to arrive in order and dont accept packetloss, so when packets get lost it will retransmit and halt communication (at least in one direction) until it has arrived. UDP will just keep sending, ok some packets get lost now and then but a good prediction algorithm makes that almost unoticable in other games. I can function in for instance EQ with maybe up to 20% pl, while in AO I can get dissed every 5 min and when I do a netstat -a and try ping one of the servers I have only 3% PL and decent pingtime.

The sync problems in AO seems to me to be related to transfer halting in one direction. For instance when it happens and I try sit, nothing happens. If I was in a fight when it happens, I still see I get hit by the mob, however if I try run I still get hit (ie on the server I'm still back at the spot with the mob while on the client I am moving somewhere else). This is when the hall of mirror effect seem to happen, I move on client but not on server, and I think server tells client what parts of dungeon to render, so it never notifies client that you have moved because on the server you are still back at the monster fighting. Of course this is speculation, would be nice if someone from funcom could comment because this is IMO no 1 gamestopper problem in AO today.

[_______________Valen]

Oh boy...not this conversation again.

Too bad the archives don't work. There were pages and pages and pages of this exact discussion back in July.

[_______________Fazun]

Originally posted by Valen
Oh boy...not this conversation again.

Too bad the archives don't work. There were pages and pages and pages of this exact discussion back in July.

lol yep and i doubt we can change anythin anyway ....

[_____________fearGod]

Originally posted by Fazun


lol yep and i doubt we can change anythin anyway ....

Which is why I will be bailing out, as soon as another MMORPG hit's the mkt.
City of Heroes or maybe even <shudder> Star Wars Galaxies.

[_______________Garzu]

Originally posted by MSag
They need to start using UDP packets like every other game instead of TCP to send the data IMO. At least make it an option. Having programmed a bit of network code, I am 90% sure that TCP is the reason its so hard for them to fix the sync problems.

Dont know **** about what you talk about... but it is sure as hell worth a try!!!

[RandaZ]

Originally posted by MSag


More or less yes, problem is TCP guarantees data to arrive in order and dont accept packetloss, so when packets get lost it will retransmit and halt communication (at least in one direction) until it has arrived. UDP will just keep sending, ok some packets get lost now and then but a good prediction algorithm makes that almost unoticable in other games. I can function in for instance EQ with maybe up to 20% pl, while in AO I can get dissed every 5 min and when I do a netstat -a and try ping one of the servers I have only 3% PL and decent pingtime.

The sync problems in AO seems to me to be related to transfer halting in one direction. For instance when it happens and I try sit, nothing happens. If I was in a fight when it happens, I still see I get hit by the mob, however if I try run I still get hit (ie on the server I'm still back at the spot with the mob while on the client I am moving somewhere else). This is when the hall of mirror effect seem to happen, I move on client but not on server, and I think server tells client what parts of dungeon to render, so it never notifies client that you have moved because on the server you are still back at the monster fighting. Of course this is speculation, would be nice if someone from funcom could comment because this is IMO no 1 gamestopper problem in AO today.

This discussion is going technical but I'm curious about that aspect so I'll add my two cents. I dunno if using UDP will actually solve the problems in AO.

Let's break down AO's networking a bit. Talking indoors specifically for now - unlike games like quake, AO does not have a static world where we can just load up the BSP tree and only send player movements through UDP.

Like you said, the server notifies the client what needs to be renders (indirectly through room description packets). Each dungeon layout is unique so these need to be sent from the server. But these packets need to be TCP since such things only really need to be sent once as we extend the active regions.

Backing up a bit, the active region is designated as the rooms that are adjacent to your current one by two. This is a design choice to avoid packet sniffing exploits (*cough*) that would otherwise expose the entire dungeon.

Anyway, that leaves us with movement updates as canidates for UDP. However, the AO client is currently automatically sending a position TCP packet (which includes velocity) periodically to the server. It also sends one every time you change your velocity vector, by turning, strafing, or even mouselook funny enough. That's more than enough for a decent prediction algorithm.

As you point out, network congestion causes loss packets on both TCP and UDP. TCP just tries resending it. Neither gets there in time anyway (or at all). If you're suffering from network congestion, the only thing that'll save you is prediction from your last position, regardless of whether it was derived from a UDP or TCP packet.

So why do some games use UDP? If we're sending a near constant stream of movement updates, it's generally more efficient. But we're not doing that since we don't need fine-grained motion. The semi-stuttering sending of AO is more than enough for the prediction algorithm to take up the slack.

You might be asking then why the AO network seems so buggy/laggy if all is okay. Well, IMHO, the deciding factor in this whole issue actually comes down to the dynamic environment used by AO. Remember, AO, unlike other games, does not load the entire level on start.

Because of the active region layout, there is no guarantee that the room you just left should be there when you get back. A little silly bit of logic, but nonetheless, AO is not static. The world as we know it, exists only in a 3x3 grid of 40m x 40m outdoors, and two adjacent rooms indoors. The server prediction algorithm will let us move to the boundary, but not cross it unless it gets verification from the client that we're actually nearing the border (at which time, the server extends the active region appropriately).

The hall of mirror effect seems to be more due to the fact that we're easily able to cross outside of the active region on the client before the server can add the new rooms to our active region. In outdoor playfields, we're warped to the server's position whenever this occurs. Inside, we're not. In fact, in a possible implementation oversight, the server just keeps ignoring your position updates when they're outside the active region. That's why you have to retrace your steps back into the active region to resynch it.

Actually, thinking about it some more, UDP might actually cause more problems. Imagine if the server loses the UDP packet that places us on the border of the active region. Effectively, we'd warp (outdoors) or desynch (indoors) more often than with TCP. Just a thought.


In anycase, to make a long story short, my perspective is that UDP is bad for AO's design. It's good for other games that are sending an almost constant stream of similar disposable packets (audio/video data comes to mind, as are quake packets). But we hardly have that situation here:

dynamic world + exploit fail-safes != UDP friendly


Of course, a lot of this is based on observation and speculation since I am not affiliated with Funcom Feel free to argue though where other places UDP might be used. Might be something I haven't thought of.


Tap

[_______________Fazun]

Originally posted by Tappert
...
Backing up a bit, the active region is designated as the rooms that are adjacent to your current one by two. This is a design choice to avoid packet sniffing exploits (*cough*) that would otherwise expose the entire dungeon.
...

hmpf ..... ehehehehe

hmm sometimes me thinks there went too much anti-cheatin into the design instead of encapsulating the data with a nice protection ....

but what do i know ehehehe

[Myga]

Originally posted by Tappert



This discussion is going technical but I'm curious about that aspect so I'll add my two cents. I dunno if using UDP will actually solve the problems in AO.

Let's break down AO's networking a bit. Talking indoors specifically for now - unlike games like quake, AO does not have a static world where we can just load up the BSP tree and only send player movements through UDP.

Like you said, the server notifies the client what needs to be renders (indirectly through room description packets). Each dungeon layout is unique so these need to be sent from the server. But these packets need to be TCP since such things only really need to be sent once as we extend the active regions.

Backing up a bit, the active region is designated as the rooms that are adjacent to your current one by two. This is a design choice to avoid packet sniffing exploits (*cough*) that would otherwise expose the entire dungeon.

Anyway, that leaves us with movement updates as canidates for UDP. However, the AO client is currently automatically sending a position TCP packet (which includes velocity) periodically to the server. It also sends one every time you change your velocity vector, by turning, strafing, or even mouselook funny enough. That's more than enough for a decent prediction algorithm.

As you point out, network congestion causes loss packets on both TCP and UDP. TCP just tries resending it. Neither gets there in time anyway (or at all). If you're suffering from network congestion, the only thing that'll save you is prediction from your last position, regardless of whether it was derived from a UDP or TCP packet.

So why do some games use UDP? If we're sending a near constant stream of movement updates, it's generally more efficient. But we're not doing that since we don't need fine-grained motion. The semi-stuttering sending of AO is more than enough for the prediction algorithm to take up the slack.

You might be asking then why the AO network seems so buggy/laggy if all is okay. Well, IMHO, the deciding factor in this whole issue actually comes down to the dynamic environment used by AO. Remember, AO, unlike other games, does not load the entire level on start.

Because of the active region layout, there is no guarantee that the room you just left should be there when you get back. A little silly bit of logic, but nonetheless, AO is not static. The world as we know it, exists only in a 3x3 grid of 40m x 40m outdoors, and two adjacent rooms indoors. The server prediction algorithm will let us move to the boundary, but not cross it unless it gets verification from the client that we're actually nearing the border (at which time, the server extends the active region appropriately).

The hall of mirror effect seems to be more due to the fact that we're easily able to cross outside of the active region on the client before the server can add the new rooms to our active region. In outdoor playfields, we're warped to the server's position whenever this occurs. Inside, we're not. In fact, in a possible implementation oversight, the server just keeps ignoring your position updates when they're outside the active region. That's why you have to retrace your steps back into the active region to resynch it.

Actually, thinking about it some more, UDP might actually cause more problems. Imagine if the server loses the UDP packet that places us on the border of the active region. Effectively, we'd warp (outdoors) or desynch (indoors) more often than with TCP. Just a thought.


In anycase, to make a long story short, my perspective is that UDP is bad for AO's design. It's good for other games that are sending an almost constant stream of similar disposable packets (audio/video data comes to mind, as are quake packets). But we hardly have that situation here:

dynamic world + exploit fail-safes != UDP friendly


Of course, a lot of this is based on observation and speculation since I am not affiliated with Funcom Feel free to argue though where other places UDP might be used. Might be something I haven't thought of.


Tap

Disagree on your conclusion here. I fail to see how dynamic world has to be tied to TCP. The world is rendered client side, the client knows the layout of the world locally (or at least it should remember the parts it has been through since it remembers the map). And player position should be overrided by the position client side, only chance you should get of beeing different places on client and server should be when no traffic whatsoever from client reaches server. Only benefit of TCP is that traffic arrives in order and nothing is lost, yet despite of that AO is the worst game I ever played when it comes to synchronization between server/client. UDP should be an option IMO.

As for exploit security with TCP your wrong. There is nothing preventing anyone from exploiting by using TCP, you can just use customize your own wsock32.dll and do whatever you want. Prolly alot more elegant solutions possible as well, I havent checked how that program who shows you destination of all missions up in mission terminal at once works. Only security you can have against exploits is good encryption and packet verification. This is totally independant of protocol used.

[_______________Lozer]

i may not be the best programer on the planit....... but here is what i see.

tcp gurentees that you get all the packes you need in the order you need them..... witch is key when loading areas with monsters or players........ the other protical would contunue play if a packet was lost.... that lost packed could represent a player or a monster. if so then the client would not render it. making it so that you do not see the monster till the server sends the packet representing you are taking damamge by a monster.


just my thoughs on the subject. ultima online use TCP, and i belaeve evercrack dose too..

[Myga]

UO might use TCP, EQ certainly dont.

Tappert has some good points about active region. EQ dont have active region, server sends info about every NPC/PC in the zone to player. Ao only sends info about PCs/NPCs in the active region. But I fail to see how it ties this game to TCP, in fact its my belief it dont, unless I can get some clarification here.

TCP today:

You fight a mob, client sends your position to server regularly. One packet get lost, client waits for confirmation from server that packet has arrived. You see yourself loosing the fight so you run. Client still waits for confirmation but gets none so wait for a timeout before resending (all other communication from client to server is HALTED until this packet has been confirmed by server). You reach last room in active region, all doors are mirrors because server never notified client that region had changed because server never got the message you moved and think you are back at the mob getting beat up.

How it could work with UDP:

You fight mob, client sends position regularly. You run, packet get lost, server assumes you stand still or move with constant speed. Next packet isnt lost, server get synchronised. Active region change on server, sends packet with notification to client. Of course this packet would need to be resent if its lost, but client could include the info about what it thinks is current active region when sending position to server, so the server know it has to resend this.

Anyways I almost quit this game when I had a week where I couldnt stay connected more than 5min without getting disconnected, fortunately the situation returned to normal. Ping to one of the servers showed a low 3% pl. How many else quit this game because of this or the "hall of mirriors" tho? Very important to fix this, because nothing is more annoying than loosing hours of xp to stuff outside your control. When that happens frequently game is not fun only frustrating and ppl leave it.

[RandaZ]

MSag,

Answering in somewhat a haphazard order, let's start with exploits.

The main problem I see with the UDP scenario is the ability for clients to spoof the server into letting the player teleport all over. That is, if the client ever dictates what the active region is directly (i.e. being placed in a non-active region as oppose to passing over a region border), then I can actually just spoof the client into sending temporary position packets throughout the entire dungeon and build a map. And for the UDP scenario to work, you need to grant the client that authority - no way around it.

It's a trade-off I know. The more authority you give the client, the better/smoother it always will appear to the player. Unfortunately, that power can always be exploited. I personally think it's interesting AO is so conservative when it comes to security. Gaute even mentioned his concerns when talking about player visibility/sneaking checks in a recent article.

Anyway, it's things like this that I was thinking of when I said dynamic world + exploit fail-safes != UDP friendly. Not that TCP inherently in itself is more secure... just that security issues dictate the network design (and indirectly the packet type). I'll break down my take on the requirements more succinctly:

Dynamic world:
a. - Client should not be required to 'know' the entire level from zoning in: active regions.
b. - Guaranteed delivery is required for the packets that update the dynamic world: extension of active regions, addition of mobs, items, etc.

Game Security:
c. - Client should only see what the player should see.
d. - Client should have no authority over the server.


Okay, this is hardly an exhaustive list of reasons, but it should be enough to illustrate. I think we'd agree that (b) needs TCP for those types of packets. It says nothing about other types of packets, like position updates. (a) and (c) are handled by the design idea of active regions and indirectly influence the choice of TCP/UDP because they need a robust mechanism to extend that active region as need be. We could use UDP, but like you point out, we have to grant the client some authority (d). TCP ain't perfect by no means, but at least we stand a better chance that we get the key packet that triggers the region extension.

Oh, for Lozer, the thing to keep in mind is that mobs and other players are only 'described' to the client at the same time an extension of the active region is made. It describes the name, geometry model, seen armor.. and entityID among other things. When the server next talks about the mob/player, it'll just send the entityID and whatever pertinent information like movement. I'm sure we've all been hit by 'invisible' mobs before. My guess is, the server forgot to send us that bit - a side issue related to active region logic.


I too wish they would fix the hall of mirrors problem. I've suggested they just let the server win and warp us back. It'll be a more jarring experience warping every so often indoors, but still way better than hitting mirror doors

I guess they could also increase the active region to three adjacent rooms, but I think that explodes the number of active rooms exponentially. I haven't done the math yet though.


On days when I'm really cynical, I think they should just drop the whole client idea, and program an asp or php page that'll let us hit a button to calculate hits on mobs. Notice I left out javascript for security reasons.

Tap

[Myga]

Well again, TCP isnt required for anything you say, you can implement all the handshaking etc TCP does in UDP if you want to. But you have the freedom to do it in a non blocking manner when using UDP, thats my main point here. TCP is just the easy way to stream data, everything is done for you but it just isnt done the optimal way for games. You just do not have the flexibility to do it in the best way for games.

[RandaZ]

That's true, you can write your own layer on-top of UDP if you want to. Let's be careful about calling it non-blocking though as that's a totally different topic. Connectionless is the proper term I think.

Anyway, semantics aside, if I was creating any other game that requires position to be very well known, for shooting or being shot at, then I'd definately use UDP to avoid the resend or out-of-order delays that you clearly pointed out.

But attacking in AO doesn't even take position into account (at least not that finely). From a design standpoint, I find it hard to label anything in AO as time-critical.


In lieu of the FC archives which we can't access, here's a similar discussion I found elsewhere that I think you might be interested in MSag:

http://www.kanga.nu/archives/MUD-Dev...read.php#00215

Just search for "UDP Revisited" for those posts. They've gone into a lot more detail than what we've done so far. Albeit, some may or may not be applicable given AO's unique structure.


On a related note, there's a post with interesting insight into the other end of the online gaming spectrum (Quake).

http://www.kanga.nu/archives/MUD-Dev...4/msg00265.php

Low-latency. What we all dream UDP can do. But I think it highlights what we generally miss when talking about UDP vs TCP. It uses a simple (actually single) packet type, is a bandwidth hog, and needs server-side buffering like no tommorrow. It's elegant in its own right and works wonders for Quake. I'm not so sure about AO though.

Anyway, there are many ways to skin a cat. IMHO, AO skins it pretty well. It might miss a patch or two but I'd be wary of any complex 'incredible machine' that claims to skin it completely. Who knows, implementation bugs in that machine might just kill the cat. Keep it simple.


Tap