Just sharing my experience of creating a website:
Here is the list of things to start
1) got to godaddy.com and search for the website address (domain name) you want and buy it.
I bought this website
2) now to start with, you can host it in a free hosting site: 200MBhost.com ,
Just register there and they will give you administrative logon details and also ftp site where you can send your website files specifically index,html(starting point of your website).
Just start with this index.html and initially have hello world kind of html tags.
You can either choose wordpress or joomla framework to build complex website.
I am using joomla which i feel suits my need , its pretty easy once you are comfortable with it.
Installation of these frameworks can be done by cPanel (Administrative panel) provided by your hosting service providers.
its not so expensive:
7.99 dollars per year for godaddy.com domain registration (It says 10 dollars , search for godaddy coupon code you will get 30% discount)
and hosting initially do it in 200MBhost.com , later you can buy hosting in hostnagar which is not so expensive and services are good enough.