Reliability - Do your homework to make sure the web host your considering has good uptime. Some promise 99% uptime meaning they guarantee your webpage will be available 99% of the time. Promises are not as good as feedback from existing customers. Search Google with the name of the company your considering along with the word uptime.
Support - If your web host is reliable then the support likely will follow. Make sure the webhost provides responsive support. Most of us dont think about support until something goes wrong, which is normal for some, however, the time to research support is prior to sign up with a web host. When your website is down your losing money and exposure, minutes can seem like days. Lots of web host are supported by a single person, although their website looks as if they have a staff. Also dont be fooled by online chat availability stating 24X7 support. Do a test and ask a question about the service and see how long it takes them to respond. This will give you an idea of thier response times.
Feautures - Choose a web host that has built in features that meet your needs upfront. For example if multiple email is important to your business, be sure to choose a host that allows flexibility in egards to email. If your going to use the web host for a software program that was written for your company of 20, then make sure the bandwith usage charges are within your range. Chances are that if this things are not cared for upfront than they will incure operations or development and implement cost later.