Is there any way I can have IE automatically configured for my proxy server? Is there some registry key?

Use the [Proxy] section of your winnt.sif

Example;

[Proxy]
FTP_Proxy_Server = http://proxyserver:80
Gopher_Proxy_Server = http://proxyserver:80
HTTP_Proxy_Server = http://proxyserver:80
Proxy_Enable = 1
Proxy_Override = <local>
Secure_Proxy_Server = http://proxyserver:80
Socks_Proxy_Server = http://proxyserver:80
Use_Same_Proxy = 1

Thanks What does proxyoverride <local> mean?

That setting is basically the one shown below "Bypass Proxy Server for Local Addressess"

Don't worry about using all off these setting you can always use

Use_Same_Proxy = 1

nice answer