The Profiler is a plugin created by the torch team. It adds tools for exploring causes of lag and is also useful to check real condition mod performance. it can be downloaded here
You can run a profiling command from the server in the chat tab or as admin in the in-game chat.
The command syntax is
!profile type --option1=xyz --option2=xzy ...
Example: !profile blocktypes --secs=20 --top=20
You can profile different parts of the game for different purposes. These types are:
|frames||Gives a profile overview of the entire game|
|session||Gives a profile overview of the session|
|entities||profiles things running in MySector|
|factions||profiles all factions of the server|
|players||profiles players, meaning everything owned by a player|
|grids||profiles the session by grids. Connected grids count as one grid|
|blocktypes||Profiles all blocks in the session by type|
|blocks||profiles all blocks in the session|
|scripts||profiles all running programmable blocks|
|physics||profiles all havok worlds|
|cleangps||cleans the GPS points set by profiling|
Most types can be run with optional commands. These change how the results get displayed, or how long the profiling will take
|--secs=N||to profile N seconds (default 10sec)|
|--top=N||to show N entries from the top (default 10)|
|--player="PlayerName"||to report values for a single player|
|--faction="FactionTag"||to report values for a single faction|
|--grid="GridName"||to report values for a specific grid|
|--this||to profile the entity you're currently controlling (players only)|
|--gps||to show positional results as GPS points (players only)|
Interpreting the result
Using the right commands, the profiler provides a quick way to identify problems in your world and quantifies them
The profiler will return the given most performance costing entries, by default 10.
Every entry will have a name and two numbers attached to it. These numbers are per tick, which is 16.66 ms at simspeed 1.0
This means that as soon as your world consumes more than 16.66ms computing time per tick, simspeed will drop.
The two numbers the profiler returns are The time (In ms = milliseconds/μs = microseconds/ns = nanoseconds) the entry took on the main thread and the amount of time this entry was updated in the tick (In upt = updates).
so our server is laggy and we want to know why...
Make sure the server is actually laggy:
Best sim: 0.9 Worst sim: 0.7 Average sim: 0.8
"Best sim" came out lower than 1.0 so the server is in fact lagging.
Let's get the general idea of what's going on:
Session: 17.17ms/f (total 4821.84ms) Network: 0.09ms/f (total 53.87ms) Replication: 0.01ms/f (total 5.04ms) GPS: 0.00ms/f (total 0.68ms) Idle: 0.00ms/f (total 0.00ms)
Game is supposed to spend no more than 16.666ms per frame but the session is eating up 17ms.
Let's go deeper and profile the session:
MySector: 13.66ms/f (total 4551.66ms) MyPhysics: 2.97ms/f (total 3321.69ms) MyProceduralWorldGenerator: 0.14ms/f (total 80.77ms) ProductionManagerCore: 0.03ms/f (total 15.85ms) MyGamePruningStructure: 0.02ms/f (total 10.90ms) RelativeTopSpeed: 0.01ms/f (total 8.41ms) MySectorWeatherComponent: 0.01ms/f (total 8.16ms) MyPlanetEnvironmentSessionComponent: 0.01ms/f (total 7.96ms) MySessionComponentSmartUpdater: 0.01ms/f (total 4.91ms)
There's an obvious elephant in the room but let's actually take a look at
0: 3 entities in 23.0km: 2.46ms/f (total 14.63ms) 1: 63 entities in 84.3km: 0.54ms/f (total 5.44ms) 2: 37 entities in 30.4km: 0.22ms/f (total 2.16ms) 3: 20 entities in 12.1km: 0.15ms/f (total 1.55ms) 4: 5 entities in 0.0km: 0.14ms/f (total 1.39ms) 5: 6 entities in 0.0km: 0.12ms/f (total 1.21ms) 6: 5 entities in 0.0km: 0.10ms/f (total 1.02ms) 7: 19 entities in 27.5km: 0.09ms/f (total 0.89ms) 8: 11 entities in 13.6km: 0.07ms/f (total 0.70ms) 9: 5 entities in 0.0km: 0.07ms/f (total 0.67ms)
Something is at 2.46ms which is unusual. Follow up with a couple more commands:
!profile physics --inspect=0to list the name of grids in the laggiest physics world,
!profile physics --takeme=0to show GPS of these grids on your screen,
!profile physics --takeme=doneto hide those GPS when you're done.
We found a piston tower stuck underground and won an easy 2ms off of the session.
Let's go back and dive deeper into
MyCubeBlock: 9.45ms/f (total 1710.17ms) MyCubeGrid: 3.06ms/f (total 690.59ms) Sandbox.Game.EntityComponents/MyIngameScriptComponent: 0.83ms/f (total 497.95ms) MyCharacter: 0.50ms/f (total 436.38ms) MySafeZone: 0.40ms/f (total 238.91ms) MyVoxelMap: 0.27ms/f (total 160.10ms) MyPlanet: 0.04ms/f (total 23.92ms) MyVoxelPhysics: 0.04ms/f (total 22.61ms)
Seems like blocks being laggy; going deeper:
AIS Albatross: 2.51ms/f (total 139.19ms) Asteroid Fortress: 2.08ms/f (total 52.20ms) AIS_Ezekiel_401: 1.78ms/f (total 51.53ms) FSH FishToasts Dopey Nyctitropism: 1.28ms/f (total 49.20ms) Yama: 1.08ms/f (total 48.45ms) Alpha Base: 1.07ms/f (total 48.23ms) TwoSnake Base 1: 0.47ms/f (total 44.67ms) UBA. Earth Defender HQ: 0.46ms/f (total 38.64ms) NewDawn Mk2: 0.35ms/f (total 34.34ms)
We can see several grids standing out.
Let's see what type of blocks are to blame:
HydrogenThrust: 3.60ms/f (total 57.11ms) OxygenGenerator: 2.10ms/f (total 57.16ms) GatlingTurret: 1.18ms/f (total 110.20ms) NpcInteriorTurret: 0.07ms/f (total 43.64ms) CryoChamber: 0.18ms/f (total 138.77ms) MediumCalibreTurret: 0.12ms/f (total 72.98ms) LargeCalibreTurret: 0.10ms/f (total 60.18ms) OxygenTank: 0.07ms/f (total 43.56ms) CockpitOpen: 0.05ms/f (total 38.57ms)
So we've come to see that H2/gas is the major culprit in our server.
With that in mind, we can take the next steps:
- Ask the Torch community the general number to aim for
- Limit the number of gas-related blocks and conveyor blocks per grid
All these numbers will help you address and fix various performance issues in your game server. Enjoy!