From my experience in the past, the top of the ladder should be placed half a block below the surface. I use g to snap to grid. This reduces the severity of the issue where I'm trying to get on or off the ladder at the top and I'm moving super slow across the opening as if the game can't figure out what I'm trying to do. I'm sure Red51 and team knows about this issue. On other games you press E (or F in this case) to grab onto a ladder. This would probably fix the problem but I still like the Minecraft style where you automatically grab a nearby ladder.
I figured out the ladder issue last night that it needs to be one block under the surface to stop detecting your avatar. All this, when playing with a simple manhole cover that needs more polishing. it's a tricky thing in general, yet neat how we're learning how to make use of it. Adapting our builds to it. I need to also remember to use 'G' to snap objects to places. Been wanting to use it, yet forgot what key it was binded to in these past few sessions.
I took a quick peak at the image before I left, missing the ladder in haste to work. I now see it and am going to try playing with it tonight.