Back when I was in high school, I made a solid wood gate for the entryway to my parents' house. We had to hang the gate off a stucco wall, using hinges similar to this. The holes we drilled crumbled, and the hinge screws weren't long enough to get a good purchase on the wood under the stucco. My dad picked up some 2-part epoxy from the local concrete supply house, and we essentially glued the screws into the holes. Last I checked, the hinges are still there. The gate has been changed, but it looks like they used the same hinges.
You might consider epoxy if the brick continues to give you problems.
It has been my experience, that my wall anchors/tapcons, seem to hold better when I drill them into mortar joint. I have had a lot of times where the brick chips and breaks. I seem to have a hard time drilling a clean hole into the brick to where anchor fits securely.
I lucked onto a solution that seems to be working quite well - based on a suggestion from Brian. The hole in the brick was not working for me and I snapped one bit trying. What I am doing is drilling into the mortar joint and then placing a few inches of braided copper wire in the hole before setting the screw. It's working great so far.