It's a nice thought, but of course you have to have some faith to deny or some religion to forsake before you can be a proper apostate, and I don't qualify. Nevertheless, all those born to parents who condone or even encourage piety were probably signed up to one religion or another before they could talk, and were not consulted; often a couple of heavies were appointed at the same time to ensure that the undertaking made on their behalf was properly carried out.
It is easy enough at a later stage, when you are old enough to think for yourself, to simply forget the commitment. You can even reject it formally if you like: the National Secular Society can provide for only £3 (including UK postage) an attractive Certificate of Debaptism which you can sign, frame and hang in some appropriate room of your house; over 100,000 have been supplied since 2007.
Alternatively, you can design and print a simple one of your own like this; I have copied the wording used by the NSS but they are decent fellows and are not likely to sue me for copyright infringement; if they do, then I will immediately cease and desist.
Such a certificate is nice to have as a modest sign of inner rationality but of course it has no legal or other significance (neither does the one supplied originally by the organisation which baptised you). Also, you have to bear in mind that your name is probably recorded somewhere, and it will not be easy to have it deleted. It is uncertain which, if any, provisions of the Data Protection Act apply to parish records, and it is unlikely that the Freedom of Information Act will help you to find out what the churches have got on you.
Those who have tried have met with a variety of responses. Generally, the replies from both the Roman Catholic and CofE authorities were courteous (the Archbishop of York promised to pray for one enquirer) but generally not very helpful. Some of the exchanges of correspondence are described here. For Catholics, excommunication is an option (you just fill in a form) but this may not achieve the removal of the original entry in their records.
I don't think I would bother too much about getting my name expunged from any baptismal databases where it may be listed: the possibility that I might one day be called upon to take up arms and defend Christendom is a very remote one.