Sorry for my late response!
If you can't open the inventory or chat, it indicates that there is most likely a bandwidth/throughput issue
If there are too many queued packets, it might prevent the game from receiving/processing other packets (typically resulting in delays for chat messages, cannot open inventory etc).
Being close to another player shouldn't result in considerably more packets... player sync is indeed based on the distance to a player (i.e the closer you are to another server, the more frequently you will receive sync packets), but that should never cause such an issue... but the game doesn't distinguish between admins or regular players 
Do you maybe use any plugins?
One thing you could try is to enter the networkstats console command when this issue occurs. If it has a high number of queued packets, it indicates that it's a clientside issue. However, if this number is always 0 (or around 0), it's probably a serverside issue (i.e the server can't keep up sending all the packets). The networkstats command also provides information about the packet loss (a high number, especially a two-digit number indicates that too many packets get lost, resulting in a slow re-request). The ping also plays a major role (ping + packet loss have a huge impact on the total throughput).
Feel free to send a report when this issue occurs (but first enter the networkstats command a few times, so the report contains this information) 