My company is starting an experiment in HD video. The primary users who are going to be editing / encoding said video don't have high end machines, mostly P4s that are just too new to replace but too old to be of any real value. The company would prefer to instead buy one massive beefy server if possible for server side encoding / rendering. I've seen server applications like this for maya, lightwave, fireworks, etc but nothing for premier or premier elements, which we are using. The only thing I can think of would be to do something like install xp64 (because premier won't install on server) on a server with loads of ram and fast disk and let the users TS into it. TS just doesn't seem ideal for this though.

Is there anyone out there who works with a lot of video that could give some tips on this?

You're right about Terminal Services not being really ideal for this kind of application. Have you contacted Adobe to see if they have a server-type application?

Isn't SilverLight being designed for this?