While the material we need may be close at hand, we forget to use it when we sit down to write, simply because we haven't actually made a list.
Here's what I suggest:

1- Inventory all your personal stories
Make a list of all the personal stories that come to mind. You don't actually need to write the stories themselves because you're so familiar with them already. A few words will do. For example: "Bitten by neighbour's dog."

2- Don't censor
Write everything down. Also, note all the anecdotes you don't want to tell in public. Just use some kind of code so no one will recognize the story. It's important to write everything down because a story you want to keep secret may lead to another story that is appropriate to tell.

3- Carry a notebook and pen
During your next conversation you'll most likely hear something that triggers a memory from your past. That's when the notebook and pen come in handy. Write the story immediately, and add it to your list later on.

A great many personal stories are stored in your memory. All you need is to find a way to retrieve them and incorporate them into a list. You can then look up a story the next time you're searching for a speech topic. Furthermore, you don't have to limit yourself to your own personal stories. Your parents, grandparents and friends also have great stories that you could add to your list.

Happy writing!