url problem

[eluser]AnishKumar20[/eluser]
I am having this problem of url..

www.mysite.com/index.php?<controller-name>

it is showing this in url which does not redirect me to the controller..
i have googled a bit and got to know that index.php can be removed by .htaccess file .
But here problem is not index.php . Here problem is that "?"
Now how can this be removed .
Does anyone here ever faced this problem ?
Can anyone explain me the cause of such a problem ?

Thanks in advance !!