I queue few songs and change the queue mode to "Stop when queue is empty" and begin playing. After the queue is empty, the gmb stops but the queue mode is changed back to "Normal mode". Can we retain the queue mode to what is has been set to and rollback to the default "Normal mode" ?

I am using gmb v1.1.9.2.

I don't plan to change that currently, did you try the "wait for more" mode ? The main difference with what you want is that it will play as soon as a song is added to the queue.
If you are willing to modify the source, you can easily add a 'keep=>1' in the stop line of %QActions in gmusicbrowser.pl (installed as /usr/bin/gmusicbrowser)
If you want I can make a very simple plugin that either add this keep=>1 or add a new mode that does that, it's very easy to do.

Thank you Quentin. The "wait for more" fits what I needed :)