The correct solution (perhaps a bit difficult given the chimney scenario) would be to purchase a few metres of fly mesh from a local hardware store, cut a piece to size for the chimney hearth which will stop the bees coming into the house.
Shape another piece of mesh into a cone with the pointed end having a hole about the size of an old-fashioned pencil and a base large enough to cover the chimney pot on top of the chimney. Use a piece of bell wire to lace the side of the cone together and presstick to secure the mesh to both the hearth and chimney pot. The bees are able to exit through the pencil-sized hole but for some reason cannot work out how to get back in. As more and more exit the chimney they will cluster around the entrance, stay for a few days before flying off to make a hive elsewhere.
I've used this method on a couple of occasions when bees have tried to set up home in a cavity wall using the airbrick as an entrance. Please note that once the bees are out it is important that the airbrick is either covered with a piece of mesh or individual U-shaped pieces are forced into each airbrick hole. Otherwise the next passing bunch of bees will smell the honey residue left by the previous bees and promptly take up residence. I've attached a rather phallic looking diagram showing the use of a cone when you have a spare bee hive. I just use the cone with no bee hive.
Another possible solution to the chimney problem would be to place a fan in the hearth pointing straight up into the chimney and leave it on at a very slow speed. The bees will not like the draught and should vacate the chimney.
Killing the bees is NOT a solution.
By Athol Brown