You should be able to delete one of your own posts. You should also be able to delete one of your comments - unless somebody has replied to it. After that, you can't delete it (though I think you can still edit it, erase the message, and put something like "Message deleted" into it). As far as I know, this is how the software is written, not something that can be adjusted in the settings somewhere.
If you created your post as a story, it should have an Edit tab at the top when you are looking at it (assuming that you are still logged in...) If you click on that Edit tab and then scroll all the way down the new edit screen you'll get, past the window that shows your post and lets you edit it, there should be a Delete button at the bottom.
If you created your post as a blog entry, and put it on the front page by checking the "Promote to Front Page" option in the "Publishing Options" menu, there will not be an Edit button if you are looking at it on the front page. (In that case, the front page just shows a view of your real post, which is actually in your blog area. To edit it, you need to click on the title front page copy to switch to the original in your blog area. That should have an Edit button.)
If you want it to be easy to look at all your posts together on one page, you should do them as blog posts, and promote them to the front page when you create them. If you don't care about that, just do them as stories or events (if you want them in the calendar). The "Promote to Front Page" check box is in the "Publishing Options" menu, which is the last item in the list under the box where you type in your post, just above the Submit button.