Thanks for the details. That definitely makes more sense.
The 0.01 is the unity update tick rate? Unless I am mistaken. The tick rate of the update phase coming out of the engine is 0.01.
Oh you update physics at a different pace. Interesting. Maybe this changed for a21 or a later version? That is great to hear. Fixed update and update phase should definitely be utilized just as you mentioned. I must have misread the code. Currently I am updating it all at one time as that seemed to be how entities were handled. I do not run updates on entities every frame though, I do however run the gmUpdate method every frame and a faster rate.
I will have to take another look. It looked like the positions, physics, animations and stats were updating every tick for entities and triggered from the 0.01 unity update phase tick. This update tick triggers the gmUpdate method. Something else may have paced that down to 0.05 as you said. It was a little confusing how it was approached. Some sort of split list approach with entity in particular threw me off.
I took over gmUpdate using harmony and isolated the entity update section that triggers from it. I do not run the entity slice and split entity update the way you had it since I was able to utilize that 0.01 unity tick rate more efficiently and it looked like it was more or less to catch up on missed entities due to running too long during the update process. Since I am aiming to keep the tick runtime under 0.01, I can run multiple updates of the entire gmUpdate method within the same time frame of 0.05 that you are currently using. When I allowed the entity to update on every single tick at the 0.01 rate, things were moving around at super sonic speeds. The animations surprisingly keep up until it runs out of ai pathing, then gets stuck running in place. This might be one reason things are getting stuck in game at rare times. High lag and the server failing to keep the path up to date, might get them stuck in place as their animations out pace it. Something out paces the data available for sure, so they run in place at that point.
By using a combination of multi thread during certain phases for the player update, I can keep their runtime under 0.01 even with larger player counts.
As you stated, you are targeting a 0.05-0.06 time frame window. Since I am aiming for 0.01 as a max runtime per tick, I can split the work across that 0.05 time frame and update specific entities or do specific tasks within each tick, so long as I aim to keep them below that 0.01 runtime which, at least in testing seems to match the update tick from unity that triggers the gmUpdate.
In testing, I found I was able to run ten ticks at 0.01 and keep the animation flow to match your current game state, ai was more in tune to player movements and pathing around them, network is updated more frequent so you get less bottlenecking with packet dumps. You have multiple ticks to work with so you can focus on chunk sync to unity which is part of that gmUpdate method and gets a bit heavy with high player counts, but this keeps chunks loading really smooth around players.
When I say I run a tick, that means running the gmUpdate method fully, however I am isolating the entity update section of gmUpdate. I run ten ticks at 0.01, total equaling 0.1 time frame approx. Only players update on tick 1 and 6. They update completely, entire list both times. That utilizes multi threading as I phased certain sections like the physics which is mostly in the EntityAlive.OnUpdateLive sections of EntityAlive types. I do this with players to keep a better consistency of the movement that other players see. PvP is dramatically improved but so are the general hitbox responses due to keeping closer in sync. I hope the netcode improvements from a21 will really boost that for us. Exciting!
The physics for non alive types are mostly in the Entity.OnUpdate. I update non players single threaded but I divide the work across four ticks so tick 2 3 4 5 are for non players. If there were 100 entity to update, 25 are updated in each tick. Spreading the ai burden. This is why I am thrilled about your updates with AI since the combination with what I am trying out, equals a lot of zombies. I don't see why 400 zombies would not be possible with 100 updated across the four ticks. That should fit inside the runtimes. Tick 7 8 9 10 are for the chunk syncs to unity.
When I made it run 12 ticks, I was able to see a momentary stutter in the animation flow of the characters vs the network updates. This was very similar to when high lag and large entity counts will stutter the updates of entity normally. You also start to see a lag in the ai response while moving around the zombies. 10 ticks seems to be the limit which is 0.1 overall so long as I keep my runtimes below 0.01 for each tick. Currently the game runtime is often going well above 0.1 when large player counts or high entity numbers exist, which not only misses your 0.05 runtime rate but starts to bottleneck the netcode with multiple events getting updating at one time on the player side. It starts to look like warping.
One other trick I attempted was sector work. Splitting the map to five sectors. Update each sector with a different core handling all the entity of that sector single threaded but simultaneous to each other running their own sectors. Then run the last sector on the main which would be the middle grounds between the sectors. This would avoid physics collisions and overlaps while spreading the work. Only issue there is it does not address bottlenecking within a single sector. It would penalize all of them and so the advantages would be reduced, but typically one core was trying to update all of them, so it has to benefit in some manner.
There are two sections if isolated to avoid overlaps, they can be multi threaded. By that I mean running just the Entity.OnUpdate as its own phase and the OnUpdateLive sections as its own phase. It is tricky due to the overrides and how it is coded into the rest of the entity update process, but it can be done. It breaks the OOP pattern though so I know what you mean. You are not going to change the code now and I wouldn't dare suggest you need to. In general, this is not something I would expect you guys to ever do from the whims of some random forum post guy haha. This is your baby, I am just letting you know what I have found in testing so far.
If you multi thread these as phases you just need to be cautious of anything spawning, despawning, unloading chunks which in turn unload the entity. Of course double check for race conditions from static lists or anything you might modify while another thread is using the data. You do not want to be running your physics/collision checks and have something change position or despawn. It will crash the engine as I have experienced a lot of. To avoid this while I am updating multi threaded, I make sure the UnloadEntities method from the World class is run inside of the main thread task queue from your thread manager class. I am also running the main thread tasks twice inside of the gmUpdate method that I took over. Once in the original spot you have it placed but another right after the entity update section so that anything I toss at the main thread queue will stay inside the frame time.
You guys have a ton of awesome tools and systems that work really well together. A+. Would love to see the job system in action. I am still learning how it functions.
Sorry for the long post. I thought it would be best to give better explanations this time