One thing that the book covers that I think this article glossed over was the fact that in sufficiently large systems there's never a single "load balancer" - instead there's many layers of load balancing systems at different levels of the stack. E.g.:
DNS load balancing -> high capacity network-level load balancing -> shared reverse HTTP proxy -> application server -> database (with a "load balancer" internal to the application server load balancing among DB replicas).
Hopefully someone who actually knows what theyre talking about will be along shortly!
> Are they charged the same as external customers or do they get a 'wholesale' rate?
Id be quite surprised if internal customers are charged a markup. Presumably the whole point in operating an internal service is that you lower the cost as much as possible for your internal customers.
> As internal clients, do they run under the same conditions as external clients? Or is there a shared internal server pool that they use?
From the above book, it seems that the hardware is largely abstracted away so most services aren't really aware of servers. I assume there's some separation between internal and external customers, but at a guess that'd largely be because of the external facing services being forks of existing internal tools that have been untangled from other internal services.
> Do they get any say in the hardware or low-level configuration of the systems they use? (ie. if someone needs ultra low latency or more storage, can they just ask Joe down the hall for a machine on a more lightly loaded network, or with bunch more RAM, for the week?)
As above, the hardware is largely abstracted away. From memory, teams usually say "we think we need ~x hrs of cpu/day, y Gbps of network,..." then there's some very clever scheduling that goes on to fit all the services on to the available hardware. There's a really good chapter on this in the above book.
> Do they have the same type of performance constraints as the ones encountered by gitlab?
Presumably it depends entirely on the software being written.
(I'm a software engineer at google who is oncall at this moment)
