Using the HTTP server capability

Can one use the HTTP server capability of Orthanc to server out other html files?

Yes, this is the purpose of the “ServeFolders” official plugins, that is part of the Orthanc distribution:
https://bitbucket.org/sjodogne/orthanc/src/default/Plugins/Samples/ServeFolders/

Sébastien-

is there any place that servers out the .dll for windows?

Sorry, there is no official pre-built Windows binaries for this plugin.

You will have to compile it by yourself:
https://orthanc.chu.ulg.ac.be/book/faq/compiling.html#compiling

I have just compiled a snapshot (debug, untested version) of the plugin. It is available at the following location:
http://www.orthanc-server.com/browse.php?path=/orthanc/snapshots

HTH,
Sébastien-